Bot directory · Social preview bot

Twitterbot

X's card fetcher: it reads your page's meta tags to render the preview card attached to posts.

OperatorX (Twitter)
CategorySocial preview bot
Respects robots.txtYes, per operator documentation
robots.txt tokenTwitterbot
User-agent stringTwitterbot/1.0

Official documentation: developer.x.com

What Twitterbot does

Twitterbot fetches shared URLs to build summary cards from your Twitter Card and Open Graph tags. X's documentation describes robots.txt support for controlling card crawling.

Why it visits your site

Someone posted your link on X.

The bigger picture: social preview bots

Preview fetchers are the most misunderstood bots in a raw log because their traffic pattern looks spammy - sudden bursts of identical fetches - while being the most human-driven traffic a bot can carry. Every burst means people are sharing your URL somewhere, and the fetch exists to make your link look good: title, image, description, rendered as a card. Their volume is a shareability signal worth watching, not noise.

Blocking them is almost always an own goal: the platform does not stop its users sharing your link, it just renders the share as a bare URL, and click-through drops accordingly. The rare legitimate reason to care is staging or private environments leaking into previews - which is an access-control problem, not a bot problem.

Should you block it?

Only if you want your links to render as bare URLs on X. Preview cards measurably improve click-through; leave it alone.

Block via robots.txt

Add these lines to the robots.txt at your site root to ask Twitterbot to stay away:

User-agent: Twitterbot
Disallow: /

Remember that robots.txt is a request, not a lock - compliance is voluntary, and enforcement happens at the server. Viz's AI Control pairs robots.txt rules with server-level 403 responses and then probes your site to verify the block is actually holding.

How to see Twitterbot traffic on your site

You have three windows onto it, each with a catch. Raw hosting access logs contain every Twitterbot request, but many managed WordPress hosts don't expose them, and when they do you are grepping text files by hand. A CDN dashboard sees the traffic at the edge, but it lives outside WordPress and speaks in totals, not in your site's terms. And JavaScript analytics - GA4 and friends - will never show it at all, because Twitterbot doesn't run scripts.

Viz's answer is the request log: filter to Twitterbot and read exactly which URLs it fetched and when, right inside wp-admin - then watch the same filter after any blocking decision to see whether the visits stopped, slowed, or kept coming.

Measure first

Is Twitterbot on your site right now? Find out in two minutes.