Bot directory · SEO tool crawler

DotBot

Moz's crawler, feeding the Link Explorer index and Domain Authority metrics.

OperatorMoz
CategorySEO tool crawler
Respects robots.txtYes, per operator documentation
robots.txt tokenDotBot
User-agent stringDotBot/1.2; +https://opensiteexplorer.org/dotbot

Official documentation: moz.com

What DotBot does

DotBot crawls to maintain Moz's link index, the basis of its widely quoted Domain Authority score. It is documented, honors robots.txt and crawl-delay, and runs at lower volume than Ahrefs or Semrush.

Why it visits your site

Link data for Moz's SEO metrics.

The bigger picture: SEO tool crawlers

SEO tool crawlers are commercial data harvesters with a twist: the product they build from your pages - link graphs, keyword data, authority metrics - is one your own marketing team may pay to use. They are consistently among the highest-volume non-search crawlers on the web, and they send you nothing directly. The trade is indirect: tolerate the crawl and the industry's measurement layer, including your own dashboards, stays complete.

These crawlers are generally well-behaved - documented, robots-compliant, honoring crawl-delay - so the decision is genuinely yours to make on resource and privacy grounds. Sites that block them typically do it to deny competitors easy backlink intelligence or to trim server load; sites that allow them are usually buying the completeness of their own tooling.

Should you block it?

Low volume makes it mostly harmless; block only if you are pruning all SEO crawlers on principle - and accept that your Moz metrics will go stale.

Block via robots.txt

Add these lines to the robots.txt at your site root to ask DotBot to stay away:

User-agent: DotBot
Disallow: /

Remember that robots.txt is a request, not a lock - compliance is voluntary, and enforcement happens at the server. Viz's AI Control pairs robots.txt rules with server-level 403 responses and then probes your site to verify the block is actually holding.

How to see DotBot traffic on your site

You have three windows onto it, each with a catch. Raw hosting access logs contain every DotBot request, but many managed WordPress hosts don't expose them, and when they do you are grepping text files by hand. A CDN dashboard sees the traffic at the edge, but it lives outside WordPress and speaks in totals, not in your site's terms. And JavaScript analytics - GA4 and friends - will never show it at all, because DotBot doesn't run scripts.

Viz's answer is the request log: filter to DotBot and read exactly which URLs it fetched and when, right inside wp-admin - then watch the same filter after any blocking decision to see whether the visits stopped, slowed, or kept coming.

Measure first

Is DotBot on your site right now? Find out in two minutes.