# ClaudeBot vs Claude-User: Anthropic's training crawler and its live fetcher

> For the complete documentation index, see [llms.txt](https://attensira.com/llms.txt).
> Every page on this site is also served as Markdown: append `.md` to any URL, or send `Accept: text/markdown`.

> ClaudeBot gathers Anthropic's training data; Claude-User reads a page when a person asks Claude. What each does, what blocking costs, and our own logs.

- Canonical page: https://attensira.com/compare/claudebot-vs-claude-user
- Family: compare
- Last updated: 2026-10-04
- Author: Karl-Gustav Kallasmaa (https://attensira.com/about)
- Claims as data: https://attensira.com/claims.json?slug=claudebot-vs-claude-user

**In short:** ClaudeBot collects web content for Anthropic's model training; blocking it excludes your future pages from training. Claude-User fetches a page when a person asks Claude something; blocking it stops Claude retrieving your content for that person. Unlike OpenAI and Perplexity, Anthropic says its user fetcher honours robots.txt, so you can block training and keep live answers.

**Readings, 5 Sep 2026 to 4 Oct 2026**

| Measure | Value |
| --- | --- |
| requests labelled ClaudeBot / Claude-User in 30 days (n=104 Anthropic requests) | ~68 / ~36 |
| requests labelled Claude-SearchBot (n=104 Anthropic requests) | 0 |
| most-fetched pages read by both; 8 only by Claude-User, 5 only by ClaudeBot (n=30 pages) | 11 of 30 |

_attensira.com AI traffic log, read over the Attensira MCP · 4 Oct 2026 · by the user agent each request claimed; none IP-verified_

## Verdict

Two Anthropic agents with different jobs and, unusually, the same respect for robots.txt. ClaudeBot gathers training data; Claude-User reads a page for a person who asked Claude. Most sites that care about being quoted should allow Claude-User even if they block ClaudeBot.

## The readings on this page

### The readings at the top

attensira.com AI traffic log, read over the Attensira MCP on 2026-10-04, by the user agent each request claimed; none IP-verified · 5 Sep 2026 to 4 Oct 2026

| Reading | Value | Sample |
| --- | --- | --- |
| requests labelled ClaudeBot / Claude-User in 30 days | ~68 / ~36 | n=104 Anthropic requests |
| requests labelled Claude-SearchBot | 0 | n=104 Anthropic requests |
| most-fetched pages read by both; 8 only by Claude-User, 5 only by ClaudeBot | 11 of 30 | n=30 pages |

### Where does ChatGPT put the two Anthropic tokens?

Asked for a list of AI crawler user agents on 4 October 2026, ChatGPT put ClaudeBot on its training blocklist and Claude-SearchBot on its allow list. Claude-User is on neither, though its own table had listed it as a separate user-requested fetcher. Quoted as it came.

Stored ChatGPT answer to “list of AI crawler user agents”, asked 4 Oct 2026 (n=1 answer, attensira.com workspace, Attensira MCP).

> For a broad AI-training block, the most important tokens are roughly: GPTBot ClaudeBot CCBot Google-Extended Applebot-Extended Bytespider Meta-ExternalAgent Amazonbot Whereas you may want to allow retrieval/search bots if your goal is to remain discoverable in AI answers: OAI-SearchBot Claude-SearchBot PerplexityBot The distinction between training, search/retrieval, and user-requested fetching is important when designing robots.txt.

Named, in order: no brand. Attensira was not named.

### How do large sites treat ClaudeBot and Claude-User?

robots.txt of the Tranco top-10,000 (list K9QPW of 2026-09-01), read by Attensira on 2026-09-03 · 3 Sep 2026 to 3 Sep 2026

| Reading | Value | Sample |
| --- | --- | --- |
| disallow ClaudeBot at the root (1,077 domains) | 13.4% | n=8,036 domains |
| disallow Claude-User at the root (438 domains) | 5.5% | n=8,036 domains |
| never mention Claude-User (7,464 domains) | 92.9% | n=8,036 domains |

### Which of our pages did each agent read?

Of the 30 attensira.com pages AI crawlers fetched most in 30 days, Claude-User reached eight that ClaudeBot did not fetch in the window, including /about. People asking Claude pull in pages the training crawl skips.

attensira.com AI traffic log, page rows of the 30 most-fetched pages, Attensira MCP, attensira.com workspace · 5 Sep 2026 to 4 Oct 2026 · labelled by the user agent each request claimed · n=8 of the 30 most-fetched pages

| Our page | ClaudeBot | Claude-User |
| --- | --- | --- |
| /contact-sales | Fetched | Fetched |
| / | Fetched | Fetched |
| /pricing | Fetched | Fetched |
| /about | Not seen | Fetched |
| /features/prompt-tracking | Fetched | Fetched |
| /features/content-optimization | Fetched | Fetched |
| /features/crawler-logs | Fetched | Fetched |
| /mcp | Fetched | Fetched |

## How do ClaudeBot and Claude-User compare?

Each cell cites a claim in the Sources section below.

| Criterion | ClaudeBot | Claude-User |
| --- | --- | --- |
| **What it is** |  |  |
| What triggers the fetch | Anthropic's own crawl | A person asking Claude a question |
| Reaches you at answer time | No · No, it crawls ahead of time | Yes · Yes, it fires during a conversation |
| Full user-agent string published | No · No, token only | No · No, token only |
| **robots.txt control** |  |  |
| robots.txt user-agent token | ClaudeBot | Claude-User |
| Honours robots.txt | Yes · Yes, per Anthropic | Yes · Yes, per Anthropic |
| What blocking it does | Future material excluded from training | Claude cannot retrieve your page for a user query |
| Ways to limit it | Disallow, or slow it with Crawl-delay | Disallow its own token |
| Address list for verification | claude.com/crawling/bots.json | The same list |
| A related control to know about | Blocking by IP can stop it reading robots.txt | Search indexing is a third token, Claude-SearchBot |
| **What we measured** |  |  |
| Top-10k domains disallowing it at the root (study of 3 Sep 2026, n=8,036) | 1,077 (13.4%) | 438 (5.5%) |
| Share of Anthropic requests to attensira.com, 30 days to 4 Oct 2026 (n=104) | About 65% (autonomous) | About 35% (person-triggered) |

## What is ClaudeBot?

Anthropic's training crawler. It collects web content that could potentially contribute to training Anthropic's generative models, and restricting it signals that future material should be excluded from training.

## What is Claude-User?

Anthropic's user-initiated fetcher. It accesses a website when an individual asks Claude a question, and Anthropic documents it as controllable through robots.txt.

## When should you choose ClaudeBot?

Block ClaudeBot when you do not want your pages in Anthropic's training data. Anthropic says restricting it signals that your future materials should be excluded from its training datasets, and it has no documented effect on live answers.

## When should you choose Claude-User?

Keep Claude-User allowed when you want Claude to read your page for the person asking. Anthropic says disabling it prevents its system retrieving your content in response to a user query, which is the moment your page could be quoted.

## When is neither the right choice?

Neither token governs Claude-SearchBot, Anthropic's third agent, which indexes content to improve search results; it has its own token. And blocking by IP address alone can stop a bot reading your robots.txt at all, which Anthropic warns makes a block less reliable, not more.

## Where does Attensira fit?

Whatever it takes to win the answer. On Growth and Business, Attensira counts which Anthropic agents fetch your pages at your edge next to what ChatGPT, Perplexity, Claude, Gemini and Google AI answer.

## What sets ClaudeBot and Claude-User apart?

- Anthropic documents its user-triggered fetcher as controllable through robots.txt; OpenAI and Perplexity say theirs may not follow it, so only here can a split policy be expected to hold on both sides.
- A blanket "block Anthropic" group also blocks Claude-User, which costs the live read at the moment a person asks Claude about you.
- Claude-SearchBot, the third token, is covered by neither rule on this page and needs its own line.

## What do ClaudeBot and Claude-User actually do?

ClaudeBot is Anthropic's crawler for model development. Anthropic says it helps enhance the utility and safety of its generative AI models by collecting web content that could potentially contribute to their training, and that restricting it signals your site's future materials should be excluded from its training datasets. It crawls on Anthropic's schedule, not because anyone asked about you.

Claude-User acts for a person. Anthropic says that when individuals ask Claude questions, it may access websites using a Claude-User agent, and that disabling it prevents Anthropic's system retrieving your content in response to a user query. It fires during a conversation: someone asks Claude about your category, Claude decides your page is worth reading, and Claude-User fetches it.

The two have nothing in common except the operator. One builds future models. The other serves a present answer. Treating them as one decision, which a blanket "block Anthropic" rule does, gives up the second to stop the first.

## Why is Anthropic's approach unusual?

Because its live fetcher honours robots.txt. Anthropic states that its bots respect do-not-crawl signals by honouring industry-standard directives in robots.txt, and that Claude-User lets site owners control which sites can be accessed through user-initiated requests. Compare that with the other two large answer engines. OpenAI says robots.txt rules may not apply to ChatGPT-User, since its fetches are made on a user's behalf. Perplexity says Perplexity-User generally ignores robots.txt for the same reason.

That makes Anthropic the cleanest case for a split policy. You can keep your content out of training with a ClaudeBot rule, keep Claude able to read pages for people with no Claude-User rule, and expect both decisions to be followed. With OpenAI and Perplexity, only the first half of that sentence is in your control.

## What about Claude-SearchBot?

Anthropic runs a third agent. Claude-SearchBot navigates the web to improve search result quality for users, and Anthropic says disabling it prevents your content being indexed for search optimisation, which may reduce your visibility in user search results. It is the closest Anthropic equivalent to OpenAI's OAI-SearchBot or PerplexityBot. If you want Claude to find your pages when it searches, this is the token that matters, and neither of the two on this page covers it.

The practical rule follows: write three decisions for Anthropic, not one. Training, search indexing and live fetching each have a token, and each answers a different question about what you want from Claude. Lists of AI crawlers handle this unevenly. ChatGPT's answer to "list of AI crawler user agents" listed all three tokens with the right purposes, then left Claude-User out of both its block and allow suggestions, which is exactly the case a reader then has to decide alone.

## What should your robots.txt say?

For a site that wants to stay out of training but be found and read by Claude:

```
User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /
```

If your concern is load rather than reuse, Anthropic supports the non-standard Crawl-delay extension and shows it under ClaudeBot. A delay slows the training crawler without excluding anything:

```
User-agent: ClaudeBot
Crawl-delay: 5
```

Anthropic adds that the rules must sit on every subdomain you want covered. A blog or docs host with its own robots.txt needs the same three groups.

## How do large sites treat the three Anthropic tokens?

On 3 September 2026 we read the robots.txt of the top 10,000 domains in Tranco list K9QPW and parsed 8,036. 1,077 (13.4%) disallow ClaudeBot at the root. Claude-User and Claude-SearchBot are each disallowed by about 5.5% (438 and 442). And 7,464 (92.9%) never mention Claude-User at all, so their wildcard group decides for them. Large sites are, on the whole, already making the split this page recommends: training out more often, live fetching and search in.

The roughly 440 sites blocking Claude-User are worth a thought. Because Anthropic honours that rule, those sites have chosen to be unreadable to Claude at the moment a person asks about them. Some will have meant it. Others will have copied a list of AI user agents into one disallow group without asking what each one does. The full table is in [our study of AI crawler access](/data/ai-crawler-robots-access).

### How do large sites treat ClaudeBot and Claude-User?

In our read of 8,036 top-10,000 domains on 3 September 2026, ClaudeBot was disallowed at the root about two and a half times as often as Claude-User. Most sites never name Claude-User at all.

**Readings, 3 Sep 2026**

| Measure | Value |
| --- | --- |
| disallow ClaudeBot at the root (1,077 domains) (n=8,036 domains) | 13.4% |
| disallow Claude-User at the root (438 domains) (n=8,036 domains) | 5.5% |
| never mention Claude-User (7,464 domains) (n=8,036 domains) | 92.9% |

_robots.txt of the Tranco top-10,000 (list K9QPW of 2026-09-01), read by Attensira · 3 Sep 2026_

## What did Anthropic's agents do on our site?

Attensira counts crawler requests at the edge on attensira.com, as it does for customers on Growth and Business. In the 30 days to 4 October 2026, 104 requests carried Anthropic user agents, across 48 pages. About 65% were autonomous crawls, ClaudeBot's pattern. About 35% were fetched because a person asked Claude something, which is Claude-User. No request was labelled Claude-SearchBot in the window.

Two things stand out. Anthropic's total is small next to OpenAI's: the same site logged 7,536 OpenAI requests in the same period. And the person-triggered share is high: about a third of Anthropic's requests were people asking Claude something that needed one of our pages, against about 3% for OpenAI's live fetcher on our site. A share of 104 moves a lot with a few requests, and every label is the user agent the request claimed, with none IP-verified, so we read it as a hint rather than a finding. But a hint that much of Claude's contact with a site happens live, at answer time, is a reason not to block Claude-User casually.

## What does blocking each one cost?

Blocking ClaudeBot costs nothing you can measure. Anthropic does not document any effect on live answers, and the effect on future models is unknowable for a single site. If your objection is to training, it is the right line to write.

Blocking Claude-User costs the live read. A person asks Claude about your category, Claude decides your page would help, and Claude cannot read it. Whatever Claude says next is built from other sources. For a company that wants to be recommended, that is the expensive mistake on this page, and it is easy to make with a copy-pasted list.

## How do you verify each agent?

Anthropic publishes the addresses its crawlers use as a machine-readable list at claude.com/crawling/bots.json. A request whose source address is on that list comes from Anthropic; the user-agent token tells you which agent. A request carrying an Anthropic token from an address not on the list is not from Anthropic. Anthropic does not publish full user-agent strings, so match on the token, not an exact string.

Anthropic adds a warning worth taking seriously: blocking by IP address alone may not work reliably, because it prevents the crawler reading your robots.txt. A bot that cannot read your file cannot obey it. Use robots.txt for the decision and the address list for verification, not the other way round.

## Why does the live fetch matter more than the crawl?

Because it is the step closest to a buyer. A training crawl affects what some future model might know about your category, through a path nobody can trace for a single site. A live fetch affects what one person is told today, by a model that decided your page was worth reading for their question.

There is a freshness argument too. A model's training data is months old by the time anyone uses it. A live fetch reads your page as it is now: the current price, the current feature list. If a buyer asks Claude about you after you changed your pricing, Claude-User is how the answer can reflect the change. Blocking it means Claude answers from whatever else it has, which may be older, and may be somebody else's description of you.

## Does any of this change what Claude says about you?

Only at the edges. Allowing Claude-User and Claude-SearchBot makes your pages readable and findable; it does not make Claude choose them. What an assistant says about your category depends on which sources it trusts for the question, and for most commercial questions those are review sites, comparison pages, documentation and trade press as much as your own site. Access is the floor. The work above the floor is publishing pages that answer the question better than the pages the assistant currently reads, and earning mentions on the pages it already trusts.

Our own readings show how far apart the two are. In the same 30 days that Anthropic's and OpenAI's agents read our pages, neither ChatGPT nor Perplexity named or cited Attensira in any of 60 answers to "list of AI crawler user agents". Being fetched is not being chosen.

## How should a team record the decision?

Write it down where the next editor will see it: in the robots.txt file itself, as comments above each group. A line saying which question the group answers, who decided and when, turns a list of user agents into a policy. Without it, the file decays: someone adds a new AI user agent to the biggest disallow group because it looks like the others, and a decision nobody made goes live on every page.

## What should you do this week?

Open your live robots.txt and search it for ClaudeBot, Claude-User, Claude-SearchBot and the older anthropic-ai token, which Anthropic's current article no longer lists. Decide each on its own question: training, search, live reading. Remove any blanket group that blocks all three because someone was worried about training, and put a dated comment above each group saying who decided and why. Then look at a week of edge logs for requests from Anthropic's published addresses, and check that the agents you allowed are arriving.

## FAQ

### Does blocking ClaudeBot stop Claude from reading my site?

No. ClaudeBot is the training crawler. When a person asks Claude a question that needs your page, Claude-User makes the fetch, and it has its own robots.txt token. Blocking ClaudeBot keeps your future pages out of training; it does not stop live fetches.

### Does Claude-User obey robots.txt?

Anthropic says its bots respect do-not-crawl signals by honouring industry-standard robots.txt directives, and that Claude-User lets site owners control which sites can be accessed through user-initiated requests. OpenAI and Perplexity say their user-triggered fetchers may not follow robots.txt.

### What is Claude-SearchBot?

Anthropic's third agent. It navigates the web to improve search result quality for users, and Anthropic says blocking it prevents your content being indexed for search optimisation, which may reduce your visibility in user search results. It has its own token, so a ClaudeBot or Claude-User rule does not cover it.

### Can I slow ClaudeBot down instead of blocking it?

Yes. Anthropic supports the non-standard Crawl-delay extension to robots.txt and gives ClaudeBot as the example. A Crawl-delay line under ClaudeBot reduces crawl load without changing what Anthropic may train on, which suits a site whose problem is server cost rather than reuse.

### How do I verify that a request came from Anthropic?

Anthropic publishes the addresses its crawlers use at claude.com/crawling/bots.json. If a request's source address is on that list, it comes from Anthropic. Do not block by address alone, though: Anthropic notes that can stop a bot reading your robots.txt in the first place.

### How often do Anthropic's agents visit a small site?

On attensira.com, 104 requests carried Anthropic user agents in the 30 days to 4 October 2026, across 48 pages: about 65% autonomous crawls and 35% fetched because a person asked Claude. That is far below OpenAI's 7,536 requests to the same site, and the labels are self-reported, so treat it as a direction.

## Sources

Each entry below is one claim on this page. `retrieved` is the date the exporter or a human last saw the source say this, and nothing more: it is not a publication date and not a promise of re-verification. `cite` is the anchor on the canonical page where this claim is rendered.

### claudebot-purpose

Anthropic documents ClaudeBot as collecting web content that could potentially contribute to training its generative AI models.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claudebot-purpose
- side: claudebot

> helps enhance the utility and safety of our generative AI models by collecting web content that could potentially contribute to their training
### claudebot-token

ClaudeBot is controlled with the robots.txt user-agent token ClaudeBot.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claudebot-token
- side: claudebot
### anthropic-robots

Anthropic states that its bots respect do-not-crawl signals by honouring industry standard directives in robots.txt.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-anthropic-robots
- side: context

> Anthropic's Bots respect 'do not crawl' signals by honoring industry standard directives in robots.txt
### claudebot-block-effect

Anthropic states that restricting ClaudeBot signals that a site's future materials should be excluded from its AI model training datasets.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claudebot-block-effect
- side: claudebot

> it signals that the site's future materials should be excluded from our AI model training datasets
### claudebot-crawldelay

Anthropic supports the non-standard Crawl-delay extension to robots.txt, shown with ClaudeBot as the example user agent.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claudebot-crawldelay
- side: claudebot

> To limit crawling activity, we support the non-standard Crawl-delay extension to robots.txt
### claude-user-purpose

Anthropic documents Claude-User as the agent that may access websites when individuals ask questions to Claude.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claude-user-purpose
- side: claude-user

> When individuals ask questions to Claude, it may access websites using a Claude-User agent
### claude-user-token

Claude-User is controlled with the robots.txt user-agent token Claude-User.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claude-user-token
- side: claude-user
### claude-user-robots

Anthropic states that Claude-User lets site owners control which sites can be accessed through user-initiated requests.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claude-user-robots
- side: claude-user

> Claude-User allows site owners to control which sites can be accessed through these user-initiated requests
### claude-user-block-effect

Anthropic states that disabling Claude-User prevents its system retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-claude-user-block-effect
- side: claude-user

> Disabling Claude-User on your site prevents our system from retrieving your content in response to a user query
### anthropic-ua-unpublished

Anthropic's crawler article names its bots by token and does not publish full user-agent strings for ClaudeBot or Claude-User.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-anthropic-ua-unpublished
- side: context
### anthropic-third-agent

Anthropic documents a third agent, Claude-SearchBot, which navigates the web to improve search result quality, and states that disabling it may reduce a site's visibility and accuracy in user search results.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-anthropic-third-agent
- side: context
### anthropic-ip-blocking

Anthropic notes that blocking by IP address alone may not work reliably, because it prevents the crawler reading your robots.txt.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-anthropic-ip-blocking
- side: context
### anthropic-ips

Anthropic publishes the addresses its crawlers use as one list at claude.com/crawling/bots.json.

- source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (support.claude.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-anthropic-ips
- side: context
### dog-anthropic-auto

In the attensira.com workspace, read through the Attensira MCP on 4 October 2026, about 65% of 104 requests with Anthropic user agents in the 30 days to 4 October 2026 (48 pages) were autonomous crawls (ClaudeBot), by the user agent each request claimed; none was IP-verified.

- source: https://docs.attensira.com/mcp/tools (docs.attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-dog-anthropic-auto
- side: claudebot
### dog-anthropic-human

In the attensira.com workspace, read through the Attensira MCP on 4 October 2026, about 35% of the same 104 Anthropic requests in the 30 days to 4 October 2026 were triggered by a person asking Claude (Claude-User).

- source: https://docs.attensira.com/mcp/tools (docs.attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-dog-anthropic-human
- side: claude-user
### study-claudebot-blocked

Of the 8,036 top-10,000 Tranco domains whose robots.txt Attensira read on 3 September 2026, 1,077 (13.4%) disallow ClaudeBot at the site root.

- source: https://attensira.com/data/ai-crawler-robots-access (attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-study-claudebot-blocked
- side: claudebot
### study-claudeuser-blocked

Of the 8,036 top-10,000 Tranco domains whose robots.txt Attensira read on 3 September 2026, 438 (5.5%) disallow Claude-User at the site root.

- source: https://attensira.com/data/ai-crawler-robots-access (attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/claudebot-vs-claude-user#claim-study-claudeuser-blocked
- side: claude-user