Bots
Google-Safety: why can't robots.txt block it?
Google-Safety is the agent Google uses for abuse-specific crawling, such as checking links posted on Google properties for malware. Google states it ignores robots.txt, and it has no robots.txt token, so there is nothing to allow or block there. Let verified requests through; a visit is not, by itself, evidence that your site has been flagged.
7 days free · card required
0requests filed as Google-Safety · 30 days
of 143,601 AI crawler requests from 7 operators
| Agent | AI requests, 30 days | attensira.com robots.txt |
|---|---|---|
| Google-Safety (this page) | 0 | Ignored |
| UptimeRobot | not counted | Wildcard |
None of the 2 had a request filed in the window. "Not counted" is off the board's AI view, not a zero.
Analyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP, and our live robots.txt · · n=143,601 AI requests
Karl-Gustav Kallasmaa, Founder & CEOLast updated Who runs Google-Safety, and what is it for?
Google runs it, and we file it under product operations. Google uses it for abuse-specific crawling, such as checking links posted on Google properties for malware. It reads no content for a search index or a model.
From Google’s documentation, read .
What would a robots.txt rule for Google-Safety do?
Nothing: the operator publishes no token for it, as the comments in this block say.
# This block has no effect. Google# states that the Google-Safety# user agent ignores robots.txt# rules, and publishes no# robots.txt token for it.## User-agent: Google-Safety# Disallow: /## Control it at the network layer# or not at all.Why is there nothing to allow or block in robots.txt?
Because Google-Safety is outside robots.txt by design. Google documents it as handling abuse-specific crawling, such as malware discovery for publicly posted links on Google properties, and states that it is, as such, unaffected by crawling preferences1. Google says it plainly: the Google-Safety user agent ignores robots.txt rules3.
It is also the one agent in Google's published crawler table with no robots.txt token. Every other row gives you a lever. This one is a fetch that happens because a link to your page exists on a Google surface, and your robots.txt is not part of the conversation.
Google's framing of the whole special-case category allows for this. It describes these crawlers as performing specific functions for Google products, including abuse-specific crawling, and says they may or may not respect robots.txt5. Google-Safety is the end of that range where the answer is no. The robots.txt block above is commented out on purpose: a live Google-Safety group looks like a control, gets reviewed like one, and does nothing.
How do I verify that a visit really came from Google-Safety?
A user-agent header is free text, so a log line naming Google-Safety is a claim until you check it.
Look up the visiting IP, confirm the hostname ends in one of the suffixes in the record, then look that hostname up and confirm it returns the same IP, or match the IP against the published ranges.
Documented among the special-case crawlers, so it resolves under the rate-limited-proxy mask on google.com. Since robots.txt cannot govern it, verification is the only mechanism a site owner has for telling this agent apart from something wearing its name.
Does a Google-Safety visit mean your site was flagged?
Google does not say so, and neither does this page. Google publishes what the agent does. It does not publish what a fetch implies about the site being fetched, and reading one from the other is guesswork. The web has many confident interpretations of this token; none traces back to anything Google wrote.
What a visit does tell you is narrow and documented: a link to your page appeared somewhere on a Google property, and Google checked it. Where Google does speak about your site's security is the Security issues report in Search Console, a channel about you rather than a log line you are interpreting. If you are worried, look there.
Should you ever block it at the network layer?
Rarely, and only after verifying. Since robots.txt cannot reach it, a firewall rule is the only lever, and blocking Google's abuse checks has no upside for a site that is not hosting abuse. The cases where a block makes sense are ones where the traffic is not Google at all.
That makes verification the main event for this agent. Google documents the special-case crawlers' reverse DNS mask as rate-limited-proxy-*.google.com and publishes their ranges in special-crawlers.json4. Reverse-resolve the address, confirm the domain is google.com, forward-resolve the hostname and check it returns the original address.
Note the trap. An allowlist built on googlebot.com, the mask for Google's common crawlers, rejects every genuine Google-Safety request. If your WAF is blocking something that calls itself Google-Safety, first check whether it fails the right mask or only the wrong one.
- Reverse DNS ends in
- *.google.com
- IP range file
- special-crawlers.json
ResearcherTranscribed from Google documentation · What user-agent string does Google-Safety send?
Match the token, never the version. A rule pinned to a version stops matching silently the next time the operator ships.
How should you handle the header?
Google publishes the User-Agent in HTTP requests as Google-Safety2: thirteen characters, no preamble, no version, no URL. Match the bare token case-insensitively for logging. Do not use it for access control. A string that short is trivial to copy, and a name that is documented as ignoring robots.txt is exactly the name a scraper would choose.
Where does Google-Safety belong in a crawler report?
In its own row, outside any "AI" or "search" bucket. In the four-way job taxonomy of the AI crawler guide, it is product operations: abuse checking, unrelated to whether you are indexed or cited. Folded into a bot-traffic total, its hits inflate a number that is supposed to tell you about crawling. Its siblings in Google's special-case group, such as APIs-Google and AdsBot-Google, are also product-operations agents with their own robots.txt behaviour.
Google-Safety
- robots.txt token
- None
- Google's robots.txt position
- Ignores robots.txt
APIs-Google
- robots.txt token
- APIs-Google
- Google's robots.txt position
- Ignores the global wildcard; must be named
AdsBot-Google
- robots.txt token
- AdsBot-Google
- Google's robots.txt position
- Ignores the global wildcard; must be named
What can Attensira show you about Google-Safety?
AI traffic, on Growth and Business, counts which pages agents fetch, per operator and declared purpose, measured at the edge, so abuse checks do not inflate your crawl numbers. It reports user agents as sent, and the reverse DNS check against the special-case mask stays yours. See crawler logs, and use the robots.txt generator for the agents robots.txt can actually reach.
Google-SafetyAgents filed beside Google-Safety
Should I allow Google-Safety in my situation?
Concrete cases, and what each one implies for this agent.
You added a Google-Safety group to robots.txt and the fetches continued.
Remove the group; it never had any effect.
Google states that the Google-Safety user agent ignores robots.txt rules, and lists no robots.txt token for it in the table where every other special-case crawler has one. A rule that cannot work is worse than no rule, because it leaves a maintainer believing a control exists.
Your WAF is blocking a client identifying itself as Google-Safety and you want to know whether that is safe.
Verify the source addresses first, then decide deliberately rather than by default.
The header is thirteen characters and anything can send it, so a block based on the string alone affects the honest agent and not the dishonest one. Because robots.txt is documented as inapplicable here, the network layer is the only place a real decision can be made — which makes getting the identity right a prerequisite rather than a nicety.
You are building a crawler report and Google-Safety hits are inflating a bot-traffic total.
Give it its own row and read it as security infrastructure, not interest.
Google documents the function as abuse-specific crawling such as malware discovery for publicly posted links on Google properties. Its volume is driven by where links to you have been posted, not by your standing in any search or AI surface, so averaging it into a total makes that total less meaningful.
What else do people ask?
What is Google-Safety?
Should I allow Google-Safety?
Can I block Google-Safety in robots.txt?
Where do these facts come from?
5 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.
Google documents that the Google-Safety user agent handles abuse-specific crawling, such as malware discovery for publicly posted links on Google properties, and that it is as such unaffected by crawling preferences.
developers.google.com · retrieved
“The Google-Safety user agent handles abuse-specific crawling, such as malware discovery for publicly posted links on Google properties.”
Google publishes Google-Safety's User-Agent in HTTP requests as "Google-Safety".
developers.google.com · retrieved
“Google-Safety”
Google states that the Google-Safety user agent ignores robots.txt rules.
developers.google.com · retrieved
“The Google-Safety user agent ignores robots.txt rules.”
Google documents the special-case crawlers' reverse DNS mask as rate-limited-proxy-***-***-***-***.google.com and publishes their IP ranges in the special-crawlers.json object.
developers.google.com · retrieved
“The special-case crawlers' reverse DNS mask matches rate-limited-proxy-***-***-***-***.google.com”
Google documents the special-case crawlers as performing specific functions for Google products, including for abuse-specific crawling or fetching, and states that they may or may not respect robots.txt rules.
developers.google.com · retrieved
“These crawlers or fetchers may or may not respect robots.txt rules.”
Want whatever it takes to win the answer?
7 days free · card required