Bot directory · Search engine crawler

Sogou Spider

The crawler for Sogou, a Chinese search engine owned by Tencent, with a long-standing reputation for heavy-handed crawling.

OperatorSogou (Tencent)
CategorySearch engine crawler
Respects robots.txtUnclear / disputed
Executes JavaScriptNo - reads raw HTML
robots.txt tokenSogou web spider
User-agent stringSogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

See what Sogou Spider sees on your site → try Peekabot, free

What Sogou Spider does

Sogou Spider indexes the web for Sogou Search, a Chinese search engine acquired by Tencent in 2021 and notable for powering search inside WeChat. Sogou says it honors robots.txt, but webmasters have repeatedly reported it fetching robots.txt and then crawling disallowed paths anyway, sometimes at aggressive rates. Its documentation page - the URL embedded in its own user-agent string - is unreachable from outside China in our testing, which does not inspire confidence. It crawls plain HTML with no documented JavaScript rendering.

Why it visits your site

Search indexing for Sogou's results and Tencent's search surfaces, most relevantly WeChat search.

The bigger picture: search engine crawlers

Search crawlers remain the one category with a decades-old, well-understood bargain: they consume crawl budget and return ranked visibility. Everything about them is comparatively mature - published IP ranges, verification methods, granular directives, webmaster consoles. Blocking them is almost always self-harm, which is precisely why they need to be cleanly separated from AI crawlers in your data: a blanket bot block that catches Googlebot costs you your organic channel.

The complication the AI era added is that search indexes now feed AI features too - AI Overviews draw on Google's index, Copilot on Bing's. Operators have answered with control tokens that split AI use from search use of the same crawl. The practical upshot: your lever for AI concerns is usually a token, not a block on the search crawler itself.

Should you block it?

If you serve a Chinese audience, Sogou is worth keeping despite the rough edges, since WeChat search visibility matters there. Otherwise, many site owners block it for its crawl volume and spotty robots.txt compliance. Given the compliance reports, be prepared to enforce the block at the server level rather than trusting robots.txt alone.

Block via robots.txt

Add these lines to the robots.txt at your site root to ask Sogou Spider to stay away:

User-agent: Sogou web spider
Disallow: /

Remember that robots.txt is a request, not a lock - compliance is voluntary, and enforcement happens at the server. Viz's AI Control pairs robots.txt rules with server-level 403 responses and then probes your site to verify the block is actually holding.

How to see Sogou Spider traffic on your site

You have three windows onto it, each with a catch. Raw hosting access logs contain every Sogou Spider request, but many managed WordPress hosts don't expose them, and when they do you are grepping text files by hand. A CDN dashboard sees the traffic at the edge, but it lives outside WordPress and speaks in totals, not in your site's terms. And JavaScript analytics - GA4 and friends - will never show it at all, because Sogou Spider doesn't run scripts.

Viz's answer is the request log: filter to Sogou Spider and read exactly which URLs it fetched and when, right inside wp-admin - then watch the same filter after any blocking decision to see whether the visits stopped, slowed, or kept coming.

Measure first

Is Sogou Spider on your site right now? Find out in two minutes.