# OAI-SearchBot vs PerplexityBot: ChatGPT's and Perplexity's search crawlers

> For the complete documentation index, see [llms.txt](https://attensira.com/llms.txt).
> Every page on this site is also served as Markdown: append `.md` to any URL, or send `Accept: text/markdown`.

> The two crawlers that put your pages into ChatGPT search and Perplexity answers: what each promises, what blocking costs, and what they fetched from us.

- Canonical page: https://attensira.com/compare/oai-searchbot-vs-perplexitybot
- Family: compare
- Last updated: 2026-10-04
- Author: Karl-Gustav Kallasmaa (https://attensira.com/about)
- Claims as data: https://attensira.com/claims.json?slug=oai-searchbot-vs-perplexitybot

**In short:** OAI-SearchBot and PerplexityBot do the same job for different engines: they fetch pages so ChatGPT search and Perplexity can show and link them. Neither is used to train foundation models. Allow both if you want to be cited. OpenAI says blocked sites are not shown in ChatGPT search; Perplexity recommends allowing its bot to appear in results.

**Readings, 5 Sep 2026 to 4 Oct 2026**

| Measure | Value |
| --- | --- |
| of our 30 most-fetched pages read by OAI-SearchBot / PerplexityBot (n=30 pages) | 30 / 28 |
| requests labelled PerplexityBot, across 331 pages | 621 |
| PerplexityBot requests in the window's last 7 days | 616 of 621 |

_attensira.com AI traffic log, read over the Attensira MCP · 4 Oct 2026 · by the user agent each request claimed; none IP-verified_

## Verdict

Two search crawlers for two answer engines, each documented as separate from model training. They are not alternatives to choose between: a site that wants to be cited allows both and verifies both.

## The readings on this page

### The readings at the top

attensira.com AI traffic log, read over the Attensira MCP on 2026-10-04, by the user agent each request claimed; none IP-verified · 5 Sep 2026 to 4 Oct 2026

| Reading | Value | Sample |
| --- | --- | --- |
| of our 30 most-fetched pages read by OAI-SearchBot / PerplexityBot | 30 / 28 | n=30 pages |
| requests labelled PerplexityBot, across 331 pages | 621 |  |
| PerplexityBot requests in the window's last 7 days | 616 of 621 |  |

### How many large sites name each crawler at all?

robots.txt of the Tranco top-10,000 (list K9QPW of 2026-09-01), read by Attensira on 2026-09-03 · 3 Sep 2026 to 3 Sep 2026

| Reading | Value | Sample |
| --- | --- | --- |
| allow OAI-SearchBot by name (335 domains) | 4.2% | n=8,036 domains |
| allow PerplexityBot by name (313 domains) | 3.9% | n=8,036 domains |
| name OAI-SearchBot at all, allow or block (802) | 10.0% | n=8,036 domains |
| name PerplexityBot at all, allow or block (968) | 12.0% | n=8,036 domains |

## How do OAI-SearchBot and PerplexityBot compare?

Each cell cites a claim in the Sources section below.

| Criterion | OAI-SearchBot | PerplexityBot |
| --- | --- | --- |
| **What it is** |  |  |
| Which answer engine it feeds | ChatGPT search | Perplexity search |
| Documented purpose | Surfacing websites in ChatGPT search results | Surfacing and linking websites in Perplexity results |
| Used to train foundation models | No · No, training is a separate token, GPTBot | No · No, stated by Perplexity |
| Published address range list | openai.com/searchbot.json | perplexity.com/perplexitybot.json |
| **What blocking it does** |  |  |
| robots.txt user-agent token | OAI-SearchBot | PerplexityBot |
| Operator states a consequence for blocking | Yes · Yes, opted-out sites are not shown | Partly · Indirectly, recommends allowing it to appear |
| How long a robots.txt change takes | About 24 hours | Up to 24 hours |
| **What we measured** |  |  |
| Top-10k domains allowing it by name (study of 3 Sep 2026, n=8,036) | 335 (4.2%) | 313 (3.9%) |
| Top-10k domains naming it at all, allow or block (n=8,036) | 802 (10.0%) | 968 (12.0%) |
| Our 30 most-fetched pages it read, 30 days to 4 Oct 2026 | 30 of 30 | 28 of 30 |

## What is OAI-SearchBot?

OpenAI's retrieval crawler. It indexes websites so they can be surfaced in ChatGPT's search features, and OpenAI states that opted-out sites will not be shown in ChatGPT search answers.

## What is PerplexityBot?

Perplexity's retrieval crawler. It is designed to surface and link websites in Perplexity search results, and Perplexity states it is not used to crawl content for foundation models.

## When should you choose OAI-SearchBot?

Prioritise OAI-SearchBot when your buyers ask ChatGPT. OpenAI states that sites opted out of it are not shown in ChatGPT search answers, which makes it the one crawler with a documented, direct cost for blocking.

## When should you choose PerplexityBot?

Prioritise PerplexityBot when your buyers research in Perplexity, which shows its sources on every answer. Perplexity recommends allowing PerplexityBot and its published addresses if you want your site to appear in its results.

## When is neither the right choice?

Neither crawler decides what a person sees when they ask the engine to open a specific page: ChatGPT-User and Perplexity-User handle those fetches, and both vendors say robots.txt may not apply to them. A firewall that blocks unknown bots can also stop both crawlers even when robots.txt allows them.

## Where does Attensira fit?

Whatever it takes to win the answer. Attensira shows which of your pages ChatGPT, Perplexity, Claude, Gemini and Google AI cite and, on Growth and Business, which crawlers fetched them.

## What sets OAI-SearchBot and PerplexityBot apart?

- OpenAI draws the training line structurally, with a separate GPTBot token; Perplexity states it inside PerplexityBot's own description.
- Perplexity asks you to permit its IP ranges as well, which is a firewall setting robots.txt cannot show; a CDN bot rule can block either crawler while the file says allow.
- Crawler volume arrives in bursts, so a one-week log sample can misread either crawler's monthly rate by several times.

## Why compare two crawlers from different companies?

Because they are the two that decide whether your page can be a cited source in the two answer engines that show sources most consistently. OAI-SearchBot fetches pages for ChatGPT's search features. PerplexityBot fetches pages for Perplexity's search results. Each is the reason a link to your site can appear under an answer, and each is documented by its operator as separate from training foundation models.

Most crawler comparisons are about one company's agents: training against search, or crawler against live fetcher. This one compares like with like across vendors. The useful question is not which to allow. If you want to be cited, you allow both. The useful questions are how each vendor describes the cost of blocking, how you verify each one, and what each actually does on a real site.

## What does each operator promise?

OpenAI describes OAI-SearchBot as the crawler used to surface websites in search results in ChatGPT's search features, and states that sites opted out of it will not be shown in ChatGPT search answers. Training sits under a different crawler, GPTBot, and OpenAI states that a site can allow OAI-SearchBot while disallowing GPTBot. The promise is structural: training and search are different tokens, so you can choose them separately.

Perplexity describes PerplexityBot as designed to surface and link websites in search results on Perplexity, and says it is not used to crawl content for AI foundation models. Its documentation adds a recommendation: to ensure your site appears in search results, allow PerplexityBot in robots.txt and permit requests from Perplexity's published IP ranges. The promise is stated rather than structural: one crawler, with a written boundary.

Both say changes take about a day. OpenAI gives about 24 hours for OAI-SearchBot; Perplexity says it may take up to 24 hours for its systems to reflect changes.

## What does blocking each one cost?

Blocking OAI-SearchBot has a documented cost: your site stops appearing in ChatGPT search answers. Blocking PerplexityBot has a cost Perplexity describes indirectly: if allowing it is how you ensure you appear, blocking it is how you ensure you may not. Neither vendor publishes how quickly an existing citation disappears after a block, and answers lag behind crawl decisions.

The less obvious cost is the one nobody writes into robots.txt. Perplexity asks site owners to permit its IP ranges, which is a firewall instruction, not a robots.txt one. A bot-management rule at a CDN that challenges unknown automated traffic can block PerplexityBot or OAI-SearchBot while robots.txt says allow. The file is a request to well-behaved crawlers; the firewall is what actually answers it.

## What do the robots.txt lines look like?

For a site that wants to be cited by both engines:

```
User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /
```

If your wildcard group already allows the root, these groups are optional, but naming them protects you from a future wildcard edit that disallows more than intended. If a section should never be quoted, disallow it in both groups rather than one, so the two engines see the same site.

## What did they actually fetch from us?

Attensira counts crawler requests at the edge on attensira.com, the same way it does for customers on Growth and Business. In the 30 days to 4 October 2026, 621 requests were labelled PerplexityBot, across 331 distinct pages. In the same window 7,536 requests carried OpenAI user agents, and about 86% of those, roughly 6,480, were classed as search indexing, which is OAI-SearchBot's job. On our site the OpenAI search crawler was the busier of the two by about ten to one.

The timing matters as much as the totals. 616 of PerplexityBot's 621 requests fell in the last 7 days of the window, so for most of the month it barely visited, then read our site in a burst. A team that sampled one quiet week would conclude Perplexity ignores them; a team that sampled the last week would overstate its monthly rate about fourfold. Read crawler volume over a month, not a week.

No request was labelled Perplexity-User in the window. That is the agent Perplexity uses when a person asks it to read a specific page, so its absence says something about how people reached us, not about PerplexityBot. Every label is the user agent the request claimed; none was IP-verified, and our documentation says we do not claim verified attribution.

## Does being crawled turn into being cited?

Not on its own, and our numbers show the gap. Over the same 30 days, across the 61 prompts we track, ChatGPT cited an attensira.com page in 154 of 2,349 answers (6.6%) and Perplexity in 85 of 2,175 (3.9%). Both crawlers had read nearly every one of our most-fetched pages. Crawling made those pages eligible; it did not make them the answer.

That is the honest limit of any crawler decision. Allowing OAI-SearchBot and PerplexityBot is the floor. Whether a page is chosen depends on whether it answers the question better than the pages the engine already trusts, which for our category are vendor sites, review sites and a handful of trade publications. You cannot robots.txt your way into an answer, only out of one.

## How do you verify each one?

Both operators publish address lists: openai.com/searchbot.json for OAI-SearchBot and perplexity.com/perplexitybot.json for PerplexityBot. Match requests on the user-agent token, then check the source address against the list. A request with the right token from the wrong address is somebody else using the name.

Refresh those lists on a schedule. If your firewall allowlists by address, a stale copy blocks the real crawler the day its ranges change, and nothing in robots.txt will tell you. The cheapest check is to look at your edge logs for a week after any change and confirm both crawlers are still arriving.

## What about the agents that fetch on a person's behalf?

Each company runs a second agent that is easy to confuse with its search crawler. OpenAI's ChatGPT-User fetches a page when a person in ChatGPT asks for it, and OpenAI says robots.txt rules may not apply to it. Perplexity's Perplexity-User visits a page when a person asks Perplexity a question that needs it, and generally ignores robots.txt because a user requested the fetch.

Those agents matter for a different reason. A live fetch is a person, right now, reading an answer that involves your page. But it is not something you can reliably grant or deny through robots.txt, and it is not what puts you in the index each engine searches. Keep the two kinds apart in your logs and your policy: crawlers you allow or block in robots.txt, live fetchers you handle, if at all, at the firewall.

## How do large sites treat the two?

On 3 September 2026 we read the robots.txt files of the top 10,000 domains in Tranco list K9QPW and parsed 8,036 of them. 467 (5.8%) disallow OAI-SearchBot at the root and 655 (8.2%) disallow PerplexityBot. Explicit allows are rarer and almost level: 335 (4.2%) allow OAI-SearchBot by name and 313 (3.9%) PerplexityBot. The large majority name neither and fall back to their wildcard group.

Our data cannot say why the block rates differ, and we will not guess at motives. What it does say is that a PerplexityBot block is more common among large sites than an OAI-SearchBot block, so if you inherited a robots.txt with one, check whether the reason still applies. The full table is in [our study of AI crawler access](/data/ai-crawler-robots-access).

### How many large sites name each crawler at all?

In our read of 8,036 top-10,000 domains on 3 September 2026, about one in ten wrote any rule for OAI-SearchBot and about one in eight for PerplexityBot. Explicit allows were rare for both. Everyone else is governed by a wildcard.

**Readings, 3 Sep 2026**

| Measure | Value |
| --- | --- |
| allow OAI-SearchBot by name (335 domains) (n=8,036 domains) | 4.2% |
| allow PerplexityBot by name (313 domains) (n=8,036 domains) | 3.9% |
| name OAI-SearchBot at all, allow or block (802) (n=8,036 domains) | 10.0% |
| name PerplexityBot at all, allow or block (968) (n=8,036 domains) | 12.0% |

_robots.txt of the Tranco top-10,000 (list K9QPW of 2026-09-01), read by Attensira · 3 Sep 2026_

## How do these two fit into the rest of a crawler policy?

They are the part of the policy that should almost always say yes. A sensible AI crawler policy has three layers. Search crawlers such as OAI-SearchBot, PerplexityBot, Googlebot and Bingbot are allowed, because they are how you get found. Training crawlers such as GPTBot, ClaudeBot, CCBot and the Google-Extended token are a values decision, made once and written down with the reason. Live fetchers are left alone unless they cause load or abuse.

Writing the policy in those layers makes the file readable by the next person who edits it. The expensive robots.txt mistakes are rarely wrong decisions; they are right decisions that a later edit undid because nobody knew why a line was there. A one-line comment above each group, saying which layer it belongs to and who decided, costs nothing and protects the two lines on this page from the next clean-up.

## When is blocking one of them the right call?

When the engine is not one your buyers use and the crawl load matters more to you than the visibility, or when you have a contractual reason to keep content out of a specific third-party surface. Those cases exist, but they are rare for a company that wants to be recommended. For most sites the right decision is to allow both, verify both, and spend the effort on the pages they fetch. Then check, once a quarter, that both are still arriving from their published ranges and that the pages they read are the ones being cited.

## FAQ

### Do I need to allow both OAI-SearchBot and PerplexityBot?

If you want to be cited by both ChatGPT search and Perplexity, yes. They are separate crawlers with separate robots.txt tokens, run by different companies, and allowing one does nothing for the other. Write a group for each, or make sure your wildcard group allows the root.

### Is either crawler used to train AI models?

Not by their operators' documentation. Perplexity states that PerplexityBot is not used to crawl content for AI foundation models. OpenAI documents training under a different crawler, GPTBot, and states that a site can allow OAI-SearchBot while disallowing GPTBot.

### What happens if I block PerplexityBot?

Perplexity does not spell out a penalty the way OpenAI does for OAI-SearchBot, but its documentation says that to ensure your site appears in search results it recommends allowing PerplexityBot and its published IP ranges. Read that as: blocked sites should not expect to appear.

### How long do robots.txt changes take for each?

Both vendors give the same figure. OpenAI says about 24 hours for OAI-SearchBot, and Perplexity says it may take up to 24 hours for its systems to reflect changes. Answers that already quote your page can lag longer.

### Which one crawls more?

On our own site, OAI-SearchBot by about ten to one. In the 30 days to 4 October 2026 attensira.com logged 621 requests labelled PerplexityBot across 331 pages, against about 86% of 7,536 OpenAI requests (roughly 6,480) classed as search indexing. Your mix will differ; measure it at your edge.

### Does being crawled mean I will be cited?

No. Crawling makes a page eligible. On attensira.com, ChatGPT cited one of our pages in 154 of 2,349 answers (6.6%) and Perplexity in 85 of 2,175 (3.9%) in the 30 days to 4 October 2026, while both crawlers read nearly all of our most-fetched pages. The gap is content and authority, not access.

## Sources

Each entry below is one claim on this page. `retrieved` is the date the exporter or a human last saw the source say this, and nothing more: it is not a publication date and not a promise of re-verification. `cite` is the anchor on the canonical page where this claim is rendered.

### oai-purpose

OpenAI documents OAI-SearchBot as used to surface websites in search results in ChatGPT's search features.

- source: https://developers.openai.com/api/docs/bots (developers.openai.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-oai-purpose
- side: oai-searchbot

> Used to surface websites in search results in ChatGPT's search features
### oai-token

OAI-SearchBot is controlled with the robots.txt user-agent token OAI-SearchBot.

- source: https://developers.openai.com/api/docs/bots (developers.openai.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-oai-token
- side: oai-searchbot
### oai-ips

OpenAI publishes OAI-SearchBot's address ranges at openai.com/searchbot.json

- source: https://developers.openai.com/api/docs/bots (developers.openai.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-oai-ips
- side: oai-searchbot
### oai-optout

OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers.

- source: https://developers.openai.com/api/docs/bots (developers.openai.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-oai-optout
- side: oai-searchbot

> Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers
### oai-training-separate

OpenAI documents training under a separate crawler and token, GPTBot, rather than under OAI-SearchBot.

- source: https://developers.openai.com/api/docs/bots (developers.openai.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-oai-training-separate
- side: oai-searchbot
### oai-surface

OAI-SearchBot feeds ChatGPT's search features.

- source: https://developers.openai.com/api/docs/bots (developers.openai.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-oai-surface
- side: oai-searchbot
### oai-propagation

OpenAI states that robots.txt changes for OAI-SearchBot take about 24 hours to be reflected.

- source: https://developers.openai.com/api/docs/bots (developers.openai.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-oai-propagation
- side: oai-searchbot
### pplx-purpose

Perplexity documents PerplexityBot as designed to surface and link websites in search results on Perplexity.

- source: https://docs.perplexity.ai/docs/resources/perplexity-crawlers (docs.perplexity.ai)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-pplx-purpose
- side: perplexitybot

> designed to surface and link websites in search results on Perplexity
### pplx-token

PerplexityBot is controlled with the robots.txt user-agent token PerplexityBot.

- source: https://docs.perplexity.ai/docs/resources/perplexity-crawlers (docs.perplexity.ai)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-pplx-token
- side: perplexitybot
### pplx-ips

Perplexity publishes PerplexityBot's address ranges at perplexity.com/perplexitybot.json

- source: https://docs.perplexity.ai/docs/resources/perplexity-crawlers (docs.perplexity.ai)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-pplx-ips
- side: perplexitybot
### pplx-allow-recommendation

Perplexity recommends allowing PerplexityBot in robots.txt and permitting its published IP ranges to ensure a site appears in its search results.

- source: https://docs.perplexity.ai/docs/resources/perplexity-crawlers (docs.perplexity.ai)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-pplx-allow-recommendation
- side: perplexitybot

> To ensure your site appears in search results, we recommend allowing PerplexityBot in your site's robots.txt file and permitting requests from our published IP ranges.
### pplx-not-training

Perplexity states that PerplexityBot is not used to crawl content for AI foundation models.

- source: https://docs.perplexity.ai/docs/resources/perplexity-crawlers (docs.perplexity.ai)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-pplx-not-training
- side: perplexitybot

> It is not used to crawl content for AI foundation models
### pplx-surface

PerplexityBot feeds search results on Perplexity.

- source: https://docs.perplexity.ai/docs/resources/perplexity-crawlers (docs.perplexity.ai)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-pplx-surface
- side: perplexitybot
### pplx-propagation

Perplexity states that it may take up to 24 hours for its systems to reflect robots.txt changes.

- source: https://docs.perplexity.ai/docs/resources/perplexity-crawlers (docs.perplexity.ai)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-pplx-propagation
- side: perplexitybot
### study-oai-allowed

Of the same 8,036 domains, 335 (4.2%) allow OAI-SearchBot by name, and 802 (10.0%) write any rule for it.

- source: https://attensira.com/data/ai-crawler-robots-access (attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-study-oai-allowed
- side: oai-searchbot
### study-pplx-allowed

Of the same 8,036 domains, 313 (3.9%) allow PerplexityBot by name, and 968 (12.0%) write any rule for it.

- source: https://attensira.com/data/ai-crawler-robots-access (attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-study-pplx-allowed
- side: perplexitybot
### dog-oai-top30

In the attensira.com workspace, read through the Attensira MCP on 4 October 2026, OAI-SearchBot was among the agents seen on all 30 of the 30 most-fetched attensira.com pages in the 30 days to 4 October 2026, by the user agent each request claimed; none was IP-verified.

- source: https://docs.attensira.com/mcp/tools (docs.attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-dog-oai-top30
- side: oai-searchbot
### dog-pplx-top30

In the attensira.com workspace, read through the Attensira MCP on 4 October 2026, PerplexityBot was among the agents seen on 28 of the 30 most-fetched attensira.com pages in the 30 days to 4 October 2026 (not /features/prompt-tracking or /privacy); its 621 requests covered 331 pages, 616 of them in the last 7 days.

- source: https://docs.attensira.com/mcp/tools (docs.attensira.com)
- retrieved: 2026-10-04
- cite: https://attensira.com/compare/oai-searchbot-vs-perplexitybot#claim-dog-pplx-top30
- side: perplexitybot