For the complete documentation index, see llms.txt. Every page on this site is also served as Markdown: append `.md` to any URL, or send `Accept: text/markdown`.
Skip to content
Attensira LogoAttensira
Book a demo

Bots

Google-InspectionTool: allow it or block it?

Google-InspectionTool is the fetcher Google's testing tools use: when you run URL Inspection in Search Console or the Rich Result Test, this agent loads your page. Allow it. Google states it has no effect on Search or other products, so blocking it changes nothing for rankings and only stops your own diagnostics from seeing the page.

Start freeBook a demo

7 days free · card required

AI traffic · attensira.com30 days to 4 Oct 2026

0requests filed as Google-InspectionTool · 30 days

of 143,601 AI crawler requests from 7 operators

SEO crawlers on the board, beside attensira.com's rule for each
AgentAI requests, 30 daysattensira.com robots.txt
Google-InspectionTool (this page)0Parent's rule
AhrefsBotnot countedBlocked
AhrefsSiteAudit0Wildcard
DataForSeoBot0Wildcard
dotbotnot countedWildcard
MJ12botnot countedWildcard
rogerbot0Wildcard
Screaming Frog SEO Spidernot countedWildcard
SemrushBotnot countedBlocked

None of the 9 had a request filed in the window. "Not counted" is off the board's AI view, not a zero.

Analyst agentAnalyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP, and our live robots.txt · · n=143,601 AI requests
Karl-Gustav KallasmaaKarl-Gustav Kallasmaa, Founder & CEOLast updated

How do I allow or block Google-InspectionTool in robots.txt?

Paste this into the robots.txt at the root of your domain. Beside it, the group attensira.com serves Google-InspectionTool today.

robots.txt · for your sitepaste-ready
# This is what fetches your page when you press Test in# Search Console. Disallowing it does not change your# rankings; it blinds your own tooling.User-agent: Google-InspectionToolAllow: / User-agent: GooglebotAllow: /
Example · a group to paste, written for the token Google-InspectionTool
attensira.com/robots.txtvia a parent token
# No group names Google-InspectionTool, so it reads the# Googlebot group.User-agent: GooglebotAllow: /Disallow: /api/
As served at attensira.com/robots.txt ·

We do not name Google-InspectionTool. It follows the Googlebot group instead (Google lists Googlebot as its second token), which allows everything except /api/.

Read the whole file, or write yours with the robots.txt generator.

Who runs Google-InspectionTool, and what is it for?

Google runs it, and we file it under product operations. Loads your page when someone runs URL Inspection in Search Console or the Rich Result Test. Google states it has no effect on Search or other products.

From Google’s documentation, read .

Should you ever block Google-InspectionTool?

Almost never. Google documents rules addressed to Google-InspectionTool as affecting Search testing tools such as the Rich Result Test and URL inspection in Search Console, and states it has no effect on Google Search or other products2. Press Test live URL and this is the agent that arrives.

So a block gains nothing for rankings and costs you your own diagnostics. It is the one agent in this directory whose traffic you generate yourself, which is why the robots.txt block above allows it explicitly instead of disallowing it.

Why does Search Console say it cannot fetch a page that loads fine?

This is the failure the token actually causes, and almost nobody causes it on purpose. The symptom: URL Inspection reports the page could not be fetched, the page loads perfectly in a browser, and Googlebot is crawling normally.

The usual cause is the header. Google's documented desktop string is Google-InspectionTool/1.0: no Chrome fragment, no platform, nothing browser-shaped3. Bot filters that allowlist Googlebot by pattern often fail to recognise it, and the rejection happens at the edge, above your application, where nobody looks.

Check three places, in this order:

  1. The robots.txt group. Is there a Disallow under Google-InspectionTool, Googlebot or * for that path?
  2. The WAF or CDN bot rules. Does the rule that admits Googlebot also admit this header? This is nearly always the answer.
  3. Application-level allowlists. Any middleware that checks user agents against a list of browsers or named crawlers.

The right fix is to allow verified Google-InspectionTool, not anything that sends the short header. The header is free text, and a string that simple is trivial to copy.

How do I verify that a visit really came from Google-InspectionTool?

A user-agent header is free text, so a log line naming Google-InspectionTool is a claim until you check it.

Look up the visiting IP, confirm the hostname ends in one of the suffixes in the record, then look that hostname up and confirm it returns the same IP, or match the IP against the published ranges.

One of Google's common crawlers, so it verifies the way Googlebot does. Do it deliberately if you run a strict allowlist: the short desktop header is exactly the shape heuristic bot filters reject, and what you admit should be the verified agent, not its header.

Why does a Googlebot rule also block your tests?

Google lists this agent's robots.txt tokens as both Google-InspectionTool and Googlebot4. A Disallow addressed to Googlebot on a path therefore also stops the testing tools working there.

People hit this on staging subdomains and private sections. The intent was to keep a section out of the index; the effect was to make it untestable. If you want a section out of Search but testable, a robots.txt rule cannot express that, because both agents answer to the Googlebot token. Use a noindex on those pages instead and leave robots.txt open for them. The wildcard behaves differently from what many people expect. Google's crawlers follow only the most specific group that matches them and never combine it with User-agent: *1. A User-agent: * Disallow: / binds this tool only while no Google-InspectionTool or Googlebot group exists; once one does, that group's rules are the only ones it reads.

Is Google-InspectionTool part of your AI or search traffic?

Neither. In the four-way job taxonomy of the AI crawler guide, it is product operations: it builds no index, feeds no model and produces no citation. A rise in its hits means your team has been testing pages. It says nothing about how Google, or any AI engine, sees you.

Keep it in its own row. A crawler report that lumps test fetches in with indexing fetches shows a Googlebot "spike" every time someone audits the site, and somebody then spends a morning explaining a number that meant nothing.

VerificationGoogle
Reverse DNS ends in
*.googlebot.com*.geo.googlebot.com
IP range file
common-crawlers.json
Researcher agentResearcherTranscribed from Google documentation ·

How should you match it in logs and allowlists?

Both published strings carry Google-InspectionTool/1.0, and the 1.0 is a version Google can increment3. Match the bare token Google-InspectionTool, case-insensitively, as a whole word. Do it in your log filters and in your CDN's allowlist, which is the one people forget.

What can Attensira show you about Google-InspectionTool?

AI traffic, on Growth and Business, counts crawler fetches per operator and declared purpose at the edge, so testing traffic does not get mistaken for indexing. It reports user agents as sent; the reverse DNS check is still yours. The question that matters for AI visibility is a different one, whether engines cite your pages, and Attensira reads that in the answers themselves every day. See crawler logs and the robots.txt generator.

User-AgentGoogle
Mozilla/5.0 (compatible; Google-InspectionTool/1.0)
Researcher agentResearcherTranscribed from Google documentation ·

Should I allow Google-InspectionTool in my situation?

Concrete cases, and what each one implies for this agent.

  • URL Inspection reports that the page could not be fetched, but the URL loads fine in your browser and Googlebot is crawling normally.

    Check whether Google-InspectionTool is being blocked, in robots.txt and at the WAF.

    Google documents Google-InspectionTool as its own agent with its own robots.txt token, so a rule or a filter can stop the testing tools while leaving indexing untouched. The desktop header "Mozilla/5.0 (compatible; Google-InspectionTool/1.0)" is short and browser-unlike, which is exactly what heuristic bot filters reject even when they allowlist Googlebot.

  • You are pruning crawler traffic and Google-InspectionTool is on the list to disallow.

    Leave it allowed unless the volume is genuinely a problem, and expect the volume not to be.

    Its fetches are triggered by someone on your side pressing a button in Search Console, so the traffic scales with your own testing rather than with Google's crawling. Disallowing it saves almost nothing and costs you the ability to diagnose the next indexing problem from inside Google's own tooling.

  • Someone proposes a blanket Googlebot Disallow on a staging subdomain and wants to keep testing it in Search Console.

    Those two goals conflict. Google lists Googlebot among this agent's tokens.

    Google documents Google-InspectionTool's robots.txt tokens as both Google-InspectionTool and Googlebot, so a Googlebot group can govern the testing fetcher too. If you need the tools to work on a path, the rule blocking Googlebot there will have to be narrowed or the testing will fail in a way that looks like a Google bug.

What else do people ask?

Why does URL Inspection say my page cannot be fetched when it loads in a browser?
Usually a bot filter. The desktop header is Mozilla/5.0 (compatible; Google-InspectionTool/1.0), with no browser-shaped detail, and WAF or CDN rules that allowlist Googlebot by pattern often reject it. Check the robots.txt group, then the WAF or CDN bot rules, then any application user-agent allowlist.
Does a Googlebot Disallow block Google-InspectionTool?
Yes. Google lists this agent's robots.txt tokens as both Google-InspectionTool and Googlebot, so a path disallowed for Googlebot cannot be tested either. That is why staging sections kept out of the index often become untestable too.
Is Google-InspectionTool an AI crawler?
No. It builds no index, feeds no model and produces no citation. It is the test fetcher for Google's own tools, and a rise in its traffic means someone has been testing pages.

Where do these facts come from?

6 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.

  1. Google documents that only one robots.txt group is valid for a particular crawler. Its crawlers follow the group with the most specific user agent that matches and ignore the others, and user-agent-specific groups are not combined with the * group.

    developers.google.com · retrieved

    “Only one group is valid for a particular crawler.”

  2. Google documents that crawling preferences addressed to the Google-InspectionTool user agent affect Search testing tools such as the Rich Result Test and URL inspection in Search Console, and that it has no effect on Google Search or other products.

    developers.google.com · retrieved

    “Crawling preferences addressed to the Google-InspectionTool user agent affect Search testing tools such as the Rich Result Test and URL inspection in Search Console.”

  3. Google documents Google-InspectionTool's desktop header as "Mozilla/5.0 (compatible; Google-InspectionTool/1.0)" and a mobile form built on the Nexus 5X Chromium preamble also ending in "(compatible; Google-InspectionTool/1.0)".

    developers.google.com · retrieved · changes often, check the source

  4. Google lists Google-InspectionTool's robots.txt user agent tokens as Google-InspectionTool and Googlebot.

    developers.google.com · retrieved

  5. Google states that its common crawlers always respect robots.txt rules for automatic crawls.

    developers.google.com · retrieved

    “always respect robots.txt rules for automatic crawls”

  6. Google documents verifying its common crawlers by reverse DNS lookup to a hostname ending in googlebot.com or geo.googlebot.com followed by a forward lookup back to the same address, or by matching the address against its published common-crawlers.json range file.

    developers.google.com · retrieved

Want whatever it takes to win the answer?

Start freeBook a demo

7 days free · card required