Bots
Google-CloudVertexBot: allow it or block it?
Google-CloudVertexBot crawls a site because its owner asked Google to: Google documents it as fetching for site owners who build Vertex AI Agents, with no effect on Search. Allow it if someone in your organisation runs such an agent on your content. It is not Google's training crawler, so it does not belong on a training opt-out list.
7 days free · card required
0requests filed as Google-CloudVertexBot · 30 days
of 143,601 AI crawler requests from 7 operators
| Agent | AI requests, 30 days | attensira.com robots.txt |
|---|---|---|
| Google-CloudVertexBot (this page) | 0 | Parent's rule |
| ImagesiftBot | 0 | Wildcard |
None of the 2 had a request filed in the window. User agents as sent, not checked against address lists.
Analyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP, and our live robots.txt · · n=143,601 AI requests
Karl-Gustav Kallasmaa, Founder & CEOLast updated How do I allow or block Google-CloudVertexBot in robots.txt?
Paste this into the robots.txt at the root of your domain. Beside it, the group attensira.com serves Google-CloudVertexBot today.
# Vertex AI Agent crawls, requested by a site owner. Name# the token explicitly: a Googlebot group can also address# it.User-agent: Google-CloudVertexBotDisallow: / User-agent: GooglebotAllow: /# No group names Google-CloudVertexBot, so it reads the# Googlebot group.User-agent: GooglebotAllow: /Disallow: /api/We do not name Google-CloudVertexBot. It follows the Googlebot group instead (Google lists Googlebot as its second token), which allows everything except /api/.
Read the whole file, or write yours with the robots.txt generator.
Who runs Google-CloudVertexBot, and what is it for?
Google runs it, and we file it under product operations. Feeds an AI product that is neither a search index nor a training corpus, by the operator's own description. Not how an assistant finds pages to cite.
From Google’s documentation, read .
Who asked Google-CloudVertexBot to crawl your site?
Probably someone on your side. Google documents that rules addressed to Google-CloudVertexBot affect crawls requested by the site owners for building Vertex AI Agents, and that it has no effect on Google Search or other products2. Vertex AI is Google Cloud's platform for building agents, and this token is how one of those agents reads a site it has been pointed at.
That turns the usual question around. For most crawlers the decision is whether an outside operator may take something. Here it is usually whether a colleague's project may read your own content: a support agent built on your documentation, a sales assistant built on your product pages, a vendor's prototype.
So when it appears and nobody on the web team recognises it, the first step is to ask around, after you have confirmed the visits are Google's.
Is Google-CloudVertexBot a training crawler?
No, and this is the mistake that puts it on the wrong lists. The name contains "Vertex", Vertex AI sits close to Gemini in most people's heads, and so the token keeps turning up on AI-training opt-out lists. It does not belong there.
Google-Extended is the documented control over whether crawled content may be used for training future generations of Gemini models7, and it is a directive that issues no requests at all. Blocking Google-CloudVertexBot instead breaks somebody's agent and leaves the training question untouched. In the four-way job taxonomy of the AI crawler guide, this is product operations: it builds nothing you can be cited from and feeds no foundation model Google documents.
How do I verify that a visit really came from Google-CloudVertexBot?
A user-agent header is free text, so a log line naming Google-CloudVertexBot is a claim until you check it.
Look up the visiting IP, confirm the hostname ends in one of the suffixes in the record, then look that hostname up and confirm it returns the same IP, or match the IP against the published ranges.
One of Google's common crawlers, so it shares Googlebot's verification path. Run it before you treat an unexpected visit as unauthorised, and before you go looking for the team that asked for the crawl: Google's crawler respects robots.txt, but an obscure token nobody has a rule for is a convenient disguise.
How do you block it without touching Search?
Name it. Google lists this agent's robots.txt tokens as both Google-CloudVertexBot and Googlebot4. The consequence is asymmetric and easy to get backwards:
Google-CloudVertexBot- Effect on Vertex AI Agent crawls
- Governed
- Effect on Google Search
- None
User-agent: Googlebot- Effect on Vertex AI Agent crawls
- Can be governed
- Effect on Google Search
- Governed
User-agent: *- Effect on Vertex AI Agent crawls
- Governed if no group names it
- Effect on Google Search
- Governed if Googlebot has no group
The robots.txt block above names the specific token and keeps a separate Googlebot group, which is the only way to separate the two decisions. Google's crawlers follow the one group with the most specific matching user agent and ignore the rest, and they never combine a named group with the wildcard1. So a Google-CloudVertexBot group replaces your User-agent: * rules for this crawler entirely: write every rule you want for it inside its own group.
If the crawl is wanted on some sections and not others, scope it. An agent built on your help centre has no reason to read your blog, and a path-level rule says so without breaking the project.
- Reverse DNS ends in
- *.googlebot.com*.geo.googlebot.com
- IP range file
- common-crawlers.json
ResearcherTranscribed from Google documentation · Agents filed beside Google-CloudVertexBot
How should you match it in logs?
Google documents Google-CloudVertexBot as the identifying substring in requests rather than publishing one fixed full header3. A substring match is therefore the documented method, not a shortcut. Match it case-insensitively, as a whole word, and keep it in its own report row. A Vertex crawl and a Search crawl mean completely different things, and averaging them hides both.
How does it differ from Google's other AI-adjacent agents?
Google runs several agents with AI in their story, and they answer different questions. Google-Agent acts for a person using one of Google's agent products. Google-GeminiNotebook fetches sources a user adds to a notebook. Google-CloudVertexBot fetches what a site owner configured an agent to read. Only Google-Extended speaks to training.
What can Attensira show you about Google-CloudVertexBot?
AI traffic, on Growth and Business, counts crawler fetches per operator and declared purpose at the edge, so an agent-building crawl appears apart from Search crawling. It reports user agents as sent; the verification step stays yours. Whether Google's AI answers cite you is read separately, every day. See crawler logs and the robots.txt generator.
Google-CloudVertexBotShould I allow Google-CloudVertexBot in my situation?
Concrete cases, and what each one implies for this agent.
Google-CloudVertexBot appears in your logs and nobody on the web team recognises it.
Ask internally before you block. Google documents these crawls as requested by the site owner.
Unlike almost every other agent in this directory, a Google-CloudVertexBot fetch is documented as something someone on your side asked for. Blocking it typically breaks a Vertex AI Agent a colleague is building against your own content, and the outage will not be traced back to a robots.txt line for weeks.
You want to stop Vertex AI Agent crawling without affecting Google Search.
Write a group naming Google-CloudVertexBot. Do not rely on a Googlebot rule to express it.
Google documents Google-CloudVertexBot as also addressable by the broader Googlebot token, so the two are not independent in the direction people assume. A specific Google-CloudVertexBot group is the only way to separate the decision, and a Googlebot Disallow would take Search with it.
You are drawing up an AI-training opt-out and someone adds Google-CloudVertexBot to the list.
Keep it out of that list. The training control is Google-Extended.
Google documents Google-CloudVertexBot as serving crawls requested by site owners for building their own Vertex AI Agents, not as a foundation-model training crawler, and publishes Google-Extended as the control over whether crawled content may be used for training future generations of Gemini models. Mixing the two produces a rule that blocks your own product and leaves training untouched.
What else do people ask?
Is Google-CloudVertexBot used to train Gemini?
Does a Googlebot rule apply to Google-CloudVertexBot?
Why is Google-CloudVertexBot in my logs when nobody asked for it?
Where do these facts come from?
7 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.
Google documents that only one robots.txt group is valid for a particular crawler. Its crawlers follow the group with the most specific user agent that matches and ignore the others, and user-agent-specific groups are not combined with the * group.
developers.google.com · retrieved
“Only one group is valid for a particular crawler.”
Google documents that crawling preferences addressed to the Google-CloudVertexBot user agent affect crawls requested by the site owners for building Vertex AI Agents, and that it has no effect on Google Search or other products.
developers.google.com · retrieved
“Crawling preferences addressed to the Google-CloudVertexBot user agent affect crawls requested by the site owners' for building Vertex AI Agents.”
Google documents Google-CloudVertexBot as the substring identifying this agent in HTTP requests rather than publishing a single fixed full header string.
developers.google.com · retrieved
Google lists Google-CloudVertexBot's robots.txt user agent tokens as Google-CloudVertexBot and Googlebot.
developers.google.com · retrieved
Google states that its common crawlers always respect robots.txt rules for automatic crawls.
developers.google.com · retrieved
“always respect robots.txt rules for automatic crawls”
Google documents verifying its common crawlers by reverse DNS lookup to a hostname ending in googlebot.com or geo.googlebot.com followed by a forward lookup back to the same address, or by matching the address against its published common-crawlers.json range file.
developers.google.com · retrieved
Google documents Google-Extended as the standalone product token publishers use to manage whether content Google crawls may be used for training future generations of Gemini models.
developers.google.com · retrieved
Want whatever it takes to win the answer?
7 days free · card required