node-fetch

What node-fetch does

node-fetch brought the browser's fetch API to Node.js and quietly sets this UA - complete with a link to its old GitHub home - when the calling code doesn't provide one. Despite running in JavaScript, it doesn't execute the JavaScript on pages it fetches; it just downloads them. It ships with no robots.txt awareness, and its author would have to add that logic deliberately.

Why it visits your site

A Node.js script or service requested your page. Serverless functions, chat-app link previews, and one-off scrapers all show up under this same string.

The bigger picture: utility & developer tools

This category is the honest miscellany of a server log: monitoring services, platform callbacks, and the default user agents of HTTP libraries. The unifying rule is that the string identifies software, not intent. A curl request might be your host's health check or a vulnerability probe; a python-requests hit might be a researcher or a scraper; a WordPress UA is one site talking to another. Reading these names as a threat list misses what they actually are - a census of the scripted web touching your site, most of it mundane, some of it yours.

robots.txt barely applies here, and not out of rudeness: monitors fetch the exact URL a customer configured, feed fetchers act on a user's subscription, and bare libraries simply ship with no robots logic at all. Blocking a default library UA filters the honest and the lazy while the malicious change one line and continue, so the effective levers are different - rate limiting by IP, watching which paths get requested, and reserving blocks for named services you can verify and have decided you don't want.

Should you block it?

Blocking the default UA filters only scripts whose authors didn't care enough to change it - a weak signal at best, since the dishonest ones already have. Newer Node code often uses the built-in fetch with no UA at all, so this string is fading anyway. Rate limit abusive IPs instead.

Block via robots.txt

Add these lines to the robots.txt at your site root to ask node-fetch to stay away:

User-agent: node-fetch
Disallow: /

Remember that robots.txt is a request, not a lock - compliance is voluntary, and enforcement happens at the server. Viz's AI Control pairs robots.txt rules with server-level 403 responses and then probes your site to verify the block is actually holding.

How to see node-fetch traffic on your site

You have three windows onto it, each with a catch. Raw hosting access logs contain every node-fetch request, but many managed WordPress hosts don't expose them, and when they do you are grepping text files by hand. A CDN dashboard sees the traffic at the edge, but it lives outside WordPress and speaks in totals, not in your site's terms. And JavaScript analytics - GA4 and friends - will never show it at all, because node-fetch doesn't run scripts.

Viz's answer is the request log: filter to node-fetch and read exactly which URLs it fetched and when, right inside wp-admin - then watch the same filter after any blocking decision to see whether the visits stopped, slowed, or kept coming.