Bots
E-commerce crawlers
The e-commerce category holds crawlers that read product and checkout pages for shopping surfaces. Storebot-Google is the one documented here: Google says it affects all surfaces of Google Shopping, including the Shopping tab in Search. It respects robots.txt, verifies like Googlebot, and is often mistyped: the token is Storebot-Google, not Google-Storebot.
7 days free · card required
- Operator
- What the fetch is for
- Product operations
- robots.txt
- Respected
- Verify a visit by
- IP ranges + reverse DNS
- attensira.com/robots.txt
- No named group
- Requests to attensira.com, 30 days
- 0 filed as Storebot-Google
ResearcherGoogle documentation, our AI traffic board (n=143,601) and robots.txt · Which e-commerce crawlers are documented?
Crawls for Google Shopping surfaces, including the Shopping tab in Search. Blocking it is a Shopping decision, not a Search one.
Keep Shopping crawling; scope checkout load by path. This group replaces the * group for it, so repeat any wildcard Disallow you still need.
User-agent: Storebot-GoogleAllow: /Disallow: /cart/What does a Storebot-Google rule change?
Your presence on Google Shopping surfaces, and nothing else Google documents. Google describes Storebot-Google as affecting all surfaces of Google Shopping and Googlebot as the crawler that builds the search index. They are separate tokens with separate effects, so a Storebot rule is a Shopping decision.
Its interest in product and purchase flows follows from that scope. If checkout traffic is a load problem, scope the token with ordinary path rules, such as a Disallow for /cart/, rather than blocking it outright.
- Reverse DNS ends in
- *.googlebot.com*.geo.googlebot.com
- IP range file
- common-crawlers.json
One of Google's common crawlers, so it shares Googlebot's verification path. Verifying matters on commerce sites, where a crawler tied to Shopping is a natural disguise for a price scraper. The Storebot-Google entry, and the method step by step: how to verify a crawler.
ResearcherTranscribed from Google documentation · Why does my Storebot rule do nothing?
Most often the token is written the wrong way round. The word order is the opposite of the Google-prefixed tokens beside it in Google's documentation, and a group for Google-Storebot silently matches nothing. Check what your live robots.txt says to the real token with the crawler URL inspector.
Shopping answers in AI assistants run on a different path. The assistants' own search crawlers, listed under AI search, are what put a product page in the index they cite from.
What else do people ask?
What is an e-commerce bot?
Does blocking Storebot-Google affect my Google rankings?
Is the token Storebot-Google or Google-Storebot?
How do I verify a Storebot-Google visit?
Where do these facts come from?
3 claims on this page, each with the page it was read from; last checked .
Google documents Storebot-Google as affecting all surfaces of Google Shopping, for example the Shopping tab in Google Search and Google Shopping.
developers.google.com · retrieved
“all surfaces of Google Shopping (for example, the Shopping tab in Google Search and Google Shopping)”
Google states that its common crawlers always respect robots.txt rules for automatic crawls.
developers.google.com · retrieved
“always respect robots.txt rules for automatic crawls”
Google documents verifying its common crawlers by reverse DNS lookup to a hostname ending in googlebot.com or geo.googlebot.com followed by a forward lookup back to the same address, or by matching the address against its published common-crawlers.json range file.
developers.google.com · retrieved
Want whatever it takes to win the answer?
7 days free · card required