Bots
OAI-SearchBot: ChatGPT's search crawler
OAI-SearchBot is the OpenAI crawler that surfaces websites in ChatGPT's search results. OpenAI states that sites opted out of it will not be shown in ChatGPT search answers, so allow it if you want ChatGPT to cite you. It is separate from GPTBot, the training crawler, and each rule is independent. Verify visits against openai.com/searchbot.json.
7 days free · card required
6,480OAI-SearchBot requests to attensira.com
Estimated: 86% of OpenAI's 7,536 requests · #1 of 4 search indexing agents
| Agent | Share of requests | Requests |
|---|---|---|
| OAI-SearchBot (this page) | ≈6,480 | |
| PerplexityBot | 621 | |
| bingbot | 541 | |
| Googlebot | 189 |
≈: an operator's requests split by the purpose each of its agents declares. User agents as sent, not checked against address lists.
Analyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP · · n=143,601 requests
Karl-Gustav Kallasmaa, Founder & CEOLast updated How do I allow or block OAI-SearchBot in robots.txt?
Paste this into the robots.txt at the root of your domain. Beside it, the group attensira.com serves OAI-SearchBot today.
# Stay findable in ChatGPT search, stay out of the# training corpus.User-agent: OAI-SearchBotAllow: / User-agent: GPTBotDisallow: /User-agent: OAI-SearchBotAllow: /Disallow: /api/We name OAI-SearchBot and allow it everywhere except /api/, the same as every AI and search crawler we allow by name.
Read the whole file, or write yours with the robots.txt generator.
Who runs OAI-SearchBot, and what is it for?
OpenAI runs it, and we file it under search indexing. Surfaces websites in ChatGPT's search results. It does not decide training, which is GPTBot's separate rule.
From OpenAI’s documentation, read . Why a training fetch and a search fetch are different decisions: the AI crawler guide.
How do OpenAI's four agents divide the work?
OpenAI publishes four agents, and OAI-SearchBot sits on the opposite side of the most important line in this directory from GPTBot:
- OAI-SearchBot: this page. Indexes for ChatGPT search, and the end of that chain is a link with your domain on it.
- GPTBot: crawls content that may be used in training foundation models.6 It takes your text and gives back nothing you can measure.
- ChatGPT-User: fetches a page because a person just asked about it.
- OAI-AdsBot: checks pages submitted as ads.
Same operator, same robots.txt file, opposite consequences. What allowing an indexing crawler costs and buys is argued in training crawlers versus search crawlers.
How do sites block OAI-SearchBot by accident?
Most sites that have disappeared from ChatGPT search never named OAI-SearchBot at all. They wrote a User-agent: * rule aimed at scrapers, and every crawler without a group of its own inherited it. The fix is a group per token, so that a later change to the wildcard or to GPTBot cannot reach this one. Our robots.txt generator assembles the whole file, and the robots.txt chapter covers the directive syntax.
Why does a version-pinned log filter fail silently?
OpenAI's header for this agent carries two version numbers, the embedded Chrome build and OpenAI's own, and both move. A filter pinned to OAI-SearchBot/1.4 stops matching the next time OpenAI ships, and nothing tells you: the traffic keeps arriving and your dashboard reports less of it than there is. Match the bare token as a whole word, case-insensitively.
How do I verify that a visit really came from OAI-SearchBot?
Check the source address against openai.com/searchbot.json. OpenAI publishes a separate file per agent rather than one shared list, so a verified address tells you which of the four agents made the request. That lets an OpenAI log analysis separate indexing from training without trusting a single header, a stronger result than most operators allow. The procedure, including the reverse-DNS method other operators use instead, is in verifying a crawler.
- IP range file
- searchbot.json
OpenAI publishes no reverse-DNS suffix for OAI-SearchBot; verify by the range file.
ResearcherTranscribed from OpenAI documentation · How often do sites block OAI-SearchBot?
Measured by us, not reported by OpenAI: the robots.txt of the 10,000 highest-ranked domains on the Tranco list for 2026-09-01. A domain that never mentions the token wrote no policy for it.
The method and every row, with the CSV.
- disallow OAI-SearchBot at the root
- 5.8%
- 467 of 8,036 domains
- allow it by name
- 4.2%
- 335 domains
- never mention it
- 90.0%
- 7,234 domains
What user-agent string does OAI-SearchBot send?
Verbatim from OpenAI’s documentation, read .
The version in this string changes; check OpenAI’s page before relying on the exact text.
What does allowing OAI-SearchBot cost?
Allowing it costs crawl load and is what lets ChatGPT search show and cite your pages. OpenAI states that sites opted out of it will not be shown in ChatGPT search answers. It does not decide training: that is GPTBot, a separate rule.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https:/Should I allow OAI-SearchBot in my situation?
Concrete cases, and what each one implies for this agent.
Your ChatGPT referrals fell off a cliff after a robots.txt change and nobody can see an error anywhere.
Check whether OAI-SearchBot is caught by a wildcard group, and give it an explicit allowed group of its own.
OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers. Nothing reports this back to you — no warning, no console message, no bounce. The traffic simply stops, and a wildcard rule written to keep training crawlers out is the usual cause.
Legal wants a policy that your content is never used to train a commercial model, and marketing wants ChatGPT to keep citing you.
Disallow GPTBot, allow OAI-SearchBot, and write the two groups separately.
OpenAI states that each setting is independent of the others, and gives allowing OAI-SearchBot while disallowing GPTBot as its own worked example. Both departments get what they asked for, and the configuration is the operator's documented intent rather than a trick.
A traffic spike claiming to be OAI-SearchBot is saturating a paginated archive and you are about to rate-limit it.
Verify the source addresses against openai.com/searchbot.json first.
The user-agent header is free text, and search-crawler tokens are the ones scrapers borrow most, precisely because sites allow them by name. If the addresses are outside OpenAI's published range you are looking at a scraper, and a robots.txt rule would have done nothing to it.
What else do people ask?
What happens if I block OAI-SearchBot?
Is OAI-SearchBot the same as GPTBot?
Does allowing OAI-SearchBot mean my content is used for training?
Where do these facts come from?
6 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.
OpenAI documents OAI-SearchBot as the agent used to surface websites in search results in ChatGPT's search features.
developers.openai.com · retrieved
“used to surface websites in search results in ChatGPT's search features”
OpenAI documents OAI-SearchBot's user agent as a Chrome-derived string ending "compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot".
developers.openai.com · retrieved · changes often, check the source
OpenAI states that sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers.
developers.openai.com · retrieved
“Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers”
OpenAI publishes the addresses OAI-SearchBot fetches from as a machine-readable file at openai.com/searchbot.json, one of four per-agent range files.
developers.openai.com · retrieved
OpenAI states that each crawler setting is independent of the others, and gives allowing OAI-SearchBot while disallowing GPTBot as its worked example.
developers.openai.com · retrieved
“a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot”
OpenAI documents GPTBot, a separate robots.txt token, as the agent used to crawl content that may be used in training its generative AI foundation models.
developers.openai.com · retrieved
Want whatever it takes to win the answer?
7 days free · card required