YouBot

What YouBot does

YouBot indexes web content for You.com's AI-powered search and chat products. The operator documents the user agent and states it respects robots.txt.

Why it visits your site

Standard AI search indexing: current, linkable content.

The bigger picture: AI search bots

AI search bots are the closest thing the AI era has to the old search bargain: they index your pages so an answer engine can cite and link them. When ChatGPT search or Perplexity answers with your page as a source, the crawl that preceded it is what made the citation possible. Referral volume from these engines is still small next to Google, but it is qualitatively different traffic - a reader who clicks through from a cited answer arrives having already been told your page is the authority on their question.

The strategic mistake to avoid is lumping these bots in with training crawlers under one blanket "block AI" rule. The major operators deliberately split their tokens so that training and search indexing can be controlled separately. A site that blocks everything AI-shaped removes itself from the one AI channel that links back - usually the opposite of what the blanket rule intended.

Should you block it?

You.com's audience is smaller than ChatGPT's or Perplexity's, but the crawler is well-behaved and cited answers can send visitors. Little reason to block unless you are closing the door on AI search across the board.

Block via robots.txt

Add these lines to the robots.txt at your site root to ask YouBot to stay away:

User-agent: YouBot
Disallow: /

Remember that robots.txt is a request, not a lock - compliance is voluntary, and enforcement happens at the server. Viz's AI Control pairs robots.txt rules with server-level 403 responses and then probes your site to verify the block is actually holding.

How to see YouBot traffic on your site

You have three windows onto it, each with a catch. Raw hosting access logs contain every YouBot request, but many managed WordPress hosts don't expose them, and when they do you are grepping text files by hand. A CDN dashboard sees the traffic at the edge, but it lives outside WordPress and speaks in totals, not in your site's terms. And JavaScript analytics - GA4 and friends - will never show it at all, because YouBot doesn't run scripts.

Viz's answer is the request log: filter to YouBot and read exactly which URLs it fetched and when, right inside wp-admin - then watch the same filter after any blocking decision to see whether the visits stopped, slowed, or kept coming.