For the complete documentation index, see llms.txt. Every page on this site is also served as Markdown: append `.md` to any URL, or send `Accept: text/markdown`.
Skip to content
Attensira LogoAttensira
Book a demo

Bots

Storebot-Google: the Shopping crawler

Storebot-Google is the Google crawler for Google Shopping surfaces, including the Shopping tab in Search. Blocking it is a Shopping decision with no documented effect on ordinary Search rankings. The token is Storebot-Google, not Google-Storebot. Allow it if you sell products; verify visits by reverse DNS to googlebot.com.

Start freeBook a demo

7 days free · card required

AI traffic · attensira.com30 days to 4 Oct 2026

0requests filed as Storebot-Google · 30 days

of 143,601 AI crawler requests from 7 operators

We do not name Storebot-Google. It falls through to the wildcard group, which allows everything except /api/.

# No group names Storebot-Google,# so it reads the wildcard group.User-agent: *Allow: /Disallow: /api/
Analyst agentAnalyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP, and our live robots.txt · · n=143,601 AI requests
Karl-Gustav KallasmaaKarl-Gustav Kallasmaa, Founder & CEOLast updated

Who runs Storebot-Google, and what is it for?

Google runs it, and we file it under product operations. Crawls for Google Shopping surfaces, including the Shopping tab in Search. Blocking it is a Shopping decision, not a Search one.

From Google’s documentation, read .

How do I allow or block Storebot-Google in robots.txt?

Paste this into the robots.txt at the root of your domain.

# Keep Shopping crawling; scope# checkout load by path. This# group replaces the * group for# it, so repeat any wildcard# Disallow you still need.User-agent: Storebot-GoogleAllow: /Disallow: /cart/

Does blocking Storebot-Google affect Search rankings?

Google documents Googlebot as the crawler that builds the search indexes and Storebot-Google as the one for Shopping surfaces.1 In the taxonomy of the AI crawler guide this is product operations, not search indexing.

That has a reporting consequence too. A rise in Storebot-Google traffic is not an AI visibility signal in either direction. Give it its own row in a crawler report instead of folding it into a bot-traffic total that people read as interest.

Why does a Google-Storebot rule do nothing?

Every neighbouring token in Google's table puts the vendor first: Google-InspectionTool, Google-CloudVertexBot, Google-Extended.3 Storebot-Google is the exception, so it is the one people write wrong from memory. A group addressed to a token no agent sends produces no error and no warning. It simply has no effect, and the symptom, "the crawler is ignoring our rules", points the wrong way. Fetch your live robots.txt and read the token back character by character.

How do I verify that a visit really came from Storebot-Google?

It is one of Google's common crawlers, so it uses the Googlebot path: reverse DNS to a hostname ending in googlebot.com or geo.googlebot.com, then a forward lookup back to the same address, or a match against common-crawlers.json.5 Do this on commerce sites in particular. A Shopping crawler is an obvious costume for a price scraper, and the header it needs is public. Allowlist on the verified address, not the string. The full procedure is in verifying a crawler.

VerificationGoogle
Reverse DNS ends in
*.googlebot.com*.geo.googlebot.com
IP range file
common-crawlers.json
Researcher agentResearcherTranscribed from Google documentation ·

What user-agent string does Storebot-Google send?

Match the token, never the version. A rule pinned to a version stops matching silently the next time the operator ships.

The version in this string changes; check Google’s page before relying on the exact text.

What does allowing Storebot-Google cost?

Allowing it costs crawl load and keeps you eligible for Google Shopping surfaces. Blocking it is a Shopping decision with no documented effect on ordinary Search rankings, so block it only if you do not sell products.

User-AgentGoogle
Mozilla/5.0 (X11; Linux x86_64; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36
Researcher agentResearcherTranscribed from Google documentation ·

Should I allow Storebot-Google in my situation?

Concrete cases, and what each one implies for this agent.

  • You wrote a Google-Storebot group and nothing changed.

    Correct the token to Storebot-Google.

    Google documents the token as Storebot-Google, with the vendor word second, unlike Google-InspectionTool and Google-CloudVertexBot in the same table. A robots.txt group addressed to a token no agent sends matches nothing and reports no error, so a typo here looks exactly like a crawler ignoring your rules.

  • Storebot-Google is hammering cart and checkout URLs and adding load at peak hours.

    Scope it with path rules rather than disallowing the token outright.

    Google documents this agent as affecting all surfaces of Google Shopping, so a whole-site Disallow is a decision to withdraw from those surfaces. Disallowing the transactional paths while leaving product pages available usually removes the load without giving up the placement.

  • A load-shedding rule at your CDN is dropping requests whose user agent does not look like a browser, and Shopping data has gone stale.

    Verify and allowlist Storebot-Google specifically.

    The published desktop string carries Storebot-Google/1.0 inside the platform block rather than in the usual trailing "compatible" clause, so pattern-based bot filters written around Googlebot's shape often do not recognise it. Allowlisting on a verified source address rather than on the header is the version of this fix that a scraper cannot borrow.

What else do people ask?

Does Storebot-Google respect robots.txt?
Yes. Google states that its common crawlers, Storebot-Google among them, always respect robots.txt rules for automatic crawls, so a group naming Storebot-Google takes effect for Shopping crawling.
Does Storebot-Google send a different header from phones?
Google documents two forms. The desktop header carries Storebot-Google/1.0 on a Linux Chrome string, and the mobile one is built on a Pixel 2 preamble and carries the same Storebot-Google/1.0. Match the token, and both forms are covered.
Why is it crawling my checkout pages?
Google documents the agent's scope as the Shopping surfaces and does not list which pages it fetches. If you see it on cart or checkout URLs and the load matters, the token can be scoped with ordinary path rules rather than blocked outright.

Where do these facts come from?

5 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.

  1. Google documents Storebot-Google as affecting all surfaces of Google Shopping, for example the Shopping tab in Google Search and Google Shopping.

    developers.google.com · retrieved

    “all surfaces of Google Shopping (for example, the Shopping tab in Google Search and Google Shopping)”

  2. Google documents Storebot-Google's desktop header as "Mozilla/5.0 (X11; Linux x86_64; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36" and a mobile form built on a Pixel 2 preamble also carrying Storebot-Google/1.0.

    developers.google.com · retrieved · changes often, check the source

  3. Google documents Storebot-Google as the robots.txt user agent token for the Google StoreBot crawler.

    developers.google.com · retrieved

  4. Google states that its common crawlers always respect robots.txt rules for automatic crawls.

    developers.google.com · retrieved

    “always respect robots.txt rules for automatic crawls”

  5. Google documents verifying its common crawlers by reverse DNS lookup to a hostname ending in googlebot.com or geo.googlebot.com followed by a forward lookup back to the same address, or by matching the address against its published common-crawlers.json range file.

    developers.google.com · retrieved

Want whatever it takes to win the answer?

Start freeBook a demo

7 days free · card required