Your content. Your rules. Verified.
Measuring AI traffic is step one. AI Control is step two: choose, bot by bot, which AI crawlers get your content - and get proof the block is actually working, not just configured.
Three layers, because bots don't all play fair.
robots.txt - the polite layer
Viz serves per-bot disallow rules in your robots.txt. Well-behaved crawlers like GPTBot and ClaudeBot respect it and stop requesting pages at all.
Server-level 403 - the enforced layer
Bots that ignore robots.txt hit a hard 403 from WordPress itself. Viz logs the request first, then denies it - so your analytics still show who tried.
The probe - the trust layer
A built-in self-probe makes real requests to your site with bot and browser user agents and compares the responses. It tells you whether your block is enforced, bypassed by a cache, or not active - and re-checks daily.
robots.txt is a request. A 403 is an answer.
robots.txt has governed crawling since 1994, and it works the way a “please keep off the grass” sign works: through reputation, not force. For the big AI operators the incentive is real - honoring the protocol is how they keep publishers, regulators, and courts at arm's length - and their compliance is generally good. But the protocol has no teeth, the long tail of scrapers knows it, and a few well-funded crawlers have been credibly reported treating disallow rules as suggestions.
So a serious blocking posture needs both halves: the protocol, because it is cheap, standard, and respected by the bots that matter most; and enforcement, because policy without consequences is a press release. Viz writes the robots.txt rules and backs them with server-level 403s for the bots that keep coming - and logs every refused attempt, so compliance stops being a matter of taking anyone's word for it.
Then it closes the loop most setups leave open: verification. A rule that sits in a config file is a belief, not a fact - especially behind full-page caches and CDNs that can serve content without your server running. The probe tests your block from the outside, the way a bot would experience it, and tells you plainly: enforced, bypassed, or inactive. For the wider decision of what to block at all, see the guide on whether you should block AI crawlers and the per-bot advice in the bot directory.
Block, allow, or wait - by business model, not vibes.
If content is your product
Publishers, course creators, paywalled sites: training crawls extract your inventory with nothing in return. Blocking training bots while keeping search and retrieval open preserves discovery without donating the archive.
If content is your marketing
SaaS and stores usually want models to know them. Being absent from AI answers is a cost, not a win. Here the move is often to allow AI bots - and watch the crawl to see which pages the machines weigh.
If you're not sure yet
Measure first. Run open for a few weeks, see which bots take how much, then decide from data. Blocking is reversible either way - and the crawl log is what gives a future licensing position its numbers.
Block training. Allow search. Or anything in between.
Blocking is per-bot across the four AI categories - training crawlers, AI search bots, AI user agents, and autonomous agents. Search engines like Googlebot are never blockable, so saying no to GPTBot can't hurt your SEO.
Per-bot switches
Allow-by-default with a blocklist you control. Block one training crawler, a whole category, or nothing - and change your mind anytime.
Log, then deny
Blocked requests are recorded before they're refused, so your dashboard keeps showing exactly who keeps knocking after being told no.
Honest about caching
Full-page caches and CDNs can serve bots without WordPress ever running. The probe detects this, and Viz gives you copy-paste .htaccess and Cloudflare rules to enforce your block at the edge - it never edits your server files behind your back.
What the first month of a block teaches you.
A block is not the end of the story; it is the start of a better dataset. In the first days, well-behaved crawlers read the new robots.txt rules and their request volume falls away - you can watch it happen in the daily trend. What remains is more interesting: the requests that keep arriving and taking 403s are a live census of who ignores your posted terms. That list, with timestamps and volumes, is exactly the evidence a licensing negotiation, an abuse report to a hosting provider, or a firewall escalation needs.
Expect second-order effects, and read them calmly. Blocking a retrieval bot can reduce AI-referred visits weeks later - the cost side of the decision showing up on schedule. Blocking a training crawler usually changes nothing visible, which is itself the point: the value it was extracting never came back as traffic. Revisit the decision quarterly with the crawl trend, the referral numbers, and your answer-engine tracking in one view - a blocking posture should be a policy you maintain, not a switch you forgot.
Blocking questions, answered plainly.
Is robots.txt legally binding?
No. It is a voluntary convention from 1994 - a posted sign, not a lock. The major AI operators publicly commit to honoring it, and compliance is how they maintain that reputation. But nothing technically stops a crawler from ignoring it, and some have been credibly reported doing so. That is why Viz treats robots.txt as the first layer, not the only one.
Can blocking AI bots hurt my Google rankings?
Not if the blocking is precise. Viz never blocks search engine crawlers - Googlebot and Bingbot are categorically excluded from blocking - so refusing GPTBot or ClaudeBot has no effect on your search indexing. The Google nuance is handled correctly too: limiting Google's AI training use goes through the Google-Extended robots token, which is separate from Googlebot.
What happens when a blocked bot keeps trying?
Viz logs the request first and then serves the 403, so refused attempts stay visible in your dashboard. That record matters: persistent post-block requests tell you which operators respect your terms and which don't - useful information in any future licensing or abuse conversation.
My site is behind Cloudflare with full-page caching. Does blocking still work?
This is exactly what the probe exists for. If your cache or CDN serves pages without WordPress running, a plugin-level 403 can be bypassed for cached URLs. The probe detects that from the outside and Viz gives you copy-paste .htaccess or Cloudflare rules to enforce the block at the edge. It never edits your server configuration behind your back.
Should I just block every AI bot to be safe?
Probably not. AI user agents and search bots often represent a human on the other end - a potential reader or buyer being actively sent your way, sometimes with a citation. Blanket blocking opts you out of that distribution along with the training. The point of per-bot, per-category control is that you don't have to choose between all and nothing.
If I block a training crawler now and unblock it later, is the damage undone?
Partly, and it is worth being honest about which part. Content already ingested into a released model stays in that model - blocking is a decision about future crawls, not a recall of past ones. What unblocking restores is forward participation: new crawls, fresh retrieval, and inclusion in whatever gets trained next. That asymmetry is an argument for deciding with data early rather than by default forever.