Applebot-Extended vs Google-Extended: two opt-outs that crawl nothing

Why compare two tokens from two companies?

Because Apple and Google solved the same problem in the same way, and the shared design is the point. Both already run a crawler the web depends on: Applebot powers search features across Apple's ecosystem, including Spotlight, Siri and Safari, and Googlebot powers Google Search. Neither company wanted publishers to choose between search and AI. So each added a token that crawls nothing and only governs how pages already fetched may be used by its models.

Apple calls Applebot-Extended a secondary user agent that gives publishers additional controls over how their content can be used, and says plainly that it does not crawl webpages. Google calls Google-Extended a standalone product token and says it has no separate HTTP user-agent string; crawling is done by Google's existing user agents. The two are mirror images, and understanding one tells you most of what you need to know about the other.

What does each token actually control?

Applebot-Extended controls whether your content may be used to train Apple's general-purpose foundation models, which power generative features across Apple products, including Apple Intelligence, Services and Developer Tools. Disallow it, and Apple says your pages can still be included in search results.

Google-Extended controls whether content Google crawls may be used to train future Gemini models and for grounding in Gemini Apps and the Vertex AI API for Gemini. Grounding means supplying content at answer time to make a response more accurate. Google says the token does not affect inclusion in Google Search and is not a ranking signal.

• Model training

  Applebot-Extended

  Apple's foundation models

  Google-Extended

  Future Gemini models

• Answer-time grounding

  Applebot-Extended

  Not described by Apple

  Google-Extended

  Gemini Apps and Vertex AI

• Search inclusion

  Applebot-Extended

  Unaffected, per Apple

  Google-Extended

  Unaffected, per Google

• Crawling

  Applebot-Extended

  None; Applebot fetches

  Google-Extended

  None; Google's crawlers fetch

The difference in scope matters. Google's token reaches answer-time use in Gemini products as well as training. Apple's documentation describes its token in terms of training. If your concern is an assistant reading your current pages to answer a question, Google's token speaks to it for Gemini; Apple's documentation does not describe an equivalent.

Why is a token that crawls nothing a good design?

Because it separates two decisions publishers want to make separately. Before these tokens existed, the only lever was the crawler itself: block Googlebot and lose Search, or allow it and accept every downstream use. A usage token lets the crawler keep fetching for search while the publisher says no to AI use. The opt-out costs nothing in search terms, which is why it spread quickly.

The trade-off is that nothing can be observed. A token that makes no requests never shows up. On attensira.com, 0 of 143,601 AI-crawler requests in the 30 days to 4 October 2026 were labelled Applebot-Extended or Google-Extended. That zero is not a finding about Apple or Google; it is what a token looks like from the server. You rely on the companies' documentation and on their honouring of it.

What should your robots.txt say?

For a site that wants to stay in Apple's and Google's search results but keep its pages out of both companies' AI use:

# AI use only. These tokens do not affect search.
User-agent: Applebot-Extended
Disallow: /

User-agent: Google-Extended
Disallow: /

Do not add Applebot or Googlebot to those groups; they are the crawlers behind search. To keep only a section out, scope each group to paths instead of the root. Both files are per host: a blog on a subdomain needs the same lines in its own robots.txt.

Comment each group with the question it answers, who decided and when. A future editor who sees a "block AI" comment above Googlebot will either break your search visibility or delete a deliberate rule, and the comment is the cheapest protection against both.

What do these tokens not do?

They also do not touch Google's search-side AI features. AI Overviews and AI Mode are Search features. Google says a page must be indexed and eligible to be shown with a snippet to appear as a supporting link in them, and names noindex, nosnippet, max-snippet and data-nosnippet as the ways to limit that. If your goal is to stay out of AI Overviews, Google-Extended is the wrong tool.

What can you measure, and what can you not?

You can measure the crawlers. Googlebot shows up under its own name: on attensira.com it made 189 requests across 80 pages in the 30 days to 4 October 2026. Applebot is harder; the AI traffic board we read classifies AI operators, and Applebot is not one of the agents it reported for our site in that window, so we have no Applebot figure to give you. If Apple matters to your buyers, check your raw server logs for its user agent and verify it the way Apple documents.

You cannot measure whether a company honours a usage token. There is no request to see and no output to inspect for a single site. What you can do is read answers. Ask the assistants your buyers use the questions they ask, on a schedule, and keep the answers. Our own workspace reads ChatGPT and Perplexity daily, so we can say what those two engines cite from attensira.com; we do not read Apple Intelligence or Gemini, and we will not publish a number for either.

That gap is worth stating plainly because it shapes the decision. A training opt-out is a statement of policy whose effect you will never see directly. Make it for the reason you hold, write the reason down, and judge your visibility by what answers say, not by what your file says.

Who should allow both tokens?

Companies that would rather be learned than protected, and the reverse for companies whose writing is the product. Nobody can measure the effect of either choice for a single site, and neither company claims one, so the decision rests on what you sell:

• Developer tool whose docs are the product

  Usual choice for both tokens

  Allow

  Why

  Being learned is distribution

• Reference publisher that wants its definitions repeated

  Usual choice for both tokens

  Allow

  Why

  Same reason

• Local business described by assistants on phones

  Usual choice for both tokens

  Allow

  Why

  Accurate descriptions help

• Publisher whose writing is its revenue

  Usual choice for both tokens

  Disallow

  Why

  Answers may replace the visit

• Licensed or paywalled content

  Usual choice for both tokens

  Disallow, plus authentication

  Why

  The token is not access control

A publisher that disallows both should still keep the search crawlers allowed, so the visit can happen when an answer links to it. For a typical B2B company the decision is usually: allow both search crawlers, decide the two tokens together and in writing, then put the effort into the pages. Visibility in AI answers is decided mostly by search-side retrieval and by third-party pages that describe your category, not by whether one company's foundation model trained on your pricing page.

Is there a reason to allow one and block the other?

Sometimes. A company whose buyers use Gemini to research suppliers might allow Google-Extended so Gemini app answers can be grounded on its current pages, while disallowing Applebot-Extended because its concern with Apple is training only. A company with a licensing position on one of the two might do the reverse. Those are specific, legitimate reasons. Write them next to the rule, and revisit both tokens whenever either company changes the scope it describes, since a scope that grows can turn yesterday's reasonable allow into something you would not have agreed to.