Internet Archive (archive.org_bot)
The nonprofit crawler preserving the web in the Wayback Machine - with a complicated history around robots.txt.
archive.org_botia_archiverOfficial documentation: archive.org
What Internet Archive (archive.org_bot) does
The Internet Archive's crawlers snapshot public pages for permanent historical preservation in the Wayback Machine. In 2017 the Archive announced it would no longer honor robots.txt for archiving in many cases, arguing preservation should not be governed by a file aimed at search engines - which is why we mark compliance unclear. Removal requests go through the Archive directly.
Why it visits your site
Preservation: keeping a historical record of your site as it changes.
The bigger picture: archivers
Archival crawlers work on a different clock from everything else in your logs. They are not building a product or answering a query; they are preserving a record, and the value of that record usually only becomes visible years later - when you need proof of what a page said, when a dispute turns on a publication date, when a dead link is the only citation left. The Wayback Machine is routinely cited in journalism, litigation, and research precisely because it crawled pages nobody thought needed preserving.
That long-horizon mission is also why archivers sit awkwardly with robots.txt, a file designed to steer commercial search engines. The Internet Archive has publicly moved away from treating robots.txt as binding for preservation. If you have content that genuinely must not be archived, the reliable path is the operator's own exclusion process, not a disallow line.
Should you block it?
Think hard before blocking a public archive - Wayback snapshots are often the only proof of what a page said, useful to you as much as anyone. If specific content must not be archived, contact the Archive for exclusion rather than relying on robots.txt.
Block via robots.txt
Add these lines to the robots.txt at your site root to ask Internet Archive (archive.org_bot) to stay away:
User-agent: ia_archiver Disallow: /
Remember that robots.txt is a request, not a lock - compliance is voluntary, and enforcement happens at the server. Viz's AI Control pairs robots.txt rules with server-level 403 responses and then probes your site to verify the block is actually holding.
How to see Internet Archive (archive.org_bot) traffic on your site
You have three windows onto it, each with a catch. Raw hosting access logs contain every Internet Archive (archive.org_bot) request, but many managed WordPress hosts don't expose them, and when they do you are grepping text files by hand. A CDN dashboard sees the traffic at the edge, but it lives outside WordPress and speaks in totals, not in your site's terms. And JavaScript analytics - GA4 and friends - will never show it at all, because Internet Archive (archive.org_bot) doesn't run scripts.
Viz's answer is the request log: filter to Internet Archive (archive.org_bot) and read exactly which URLs it fetched and when, right inside wp-admin - then watch the same filter after any blocking decision to see whether the visits stopped, slowed, or kept coming.