Bots
Kagibot: Kagi's search crawler
Kagibot is the web crawler for Kagi, a paid search engine. Its job is search indexing, so allowing it is how your pages appear in Kagi results. It respects robots.txt, and when nothing names Kagibot it follows your Googlebot rules. Kagi publishes the four addresses it crawls from, so a visit can be verified.
7 days free · card required
0requests filed as Kagibot · 30 days
of 143,601 AI crawler requests from 7 operators
| Agent | AI requests, 30 days | attensira.com robots.txt |
|---|---|---|
| Kagibot (this page) | 0 | Wildcard |
| bingbot | 541 | Allowed |
| Googlebot | 189 | Allowed |
| Applebot | 0 | Allowed |
| Googlebot-Image | 0 | Parent's rule |
| Googlebot-Video | 0 | Parent's rule |
| GoogleOther | 0 | Allowed |
| GoogleOther-Image | 0 | Parent's rule |
| PetalBot | 0 | Wildcard |
2 of 9 had requests filed in the window. User agents as sent, not checked against address lists.
Analyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP, and our live robots.txt · · n=143,601 AI requests
Karl-Gustav Kallasmaa, Founder & CEOLast updated How do I allow or block Kagibot in robots.txt?
Paste this into the robots.txt at the root of your domain. Beside it, the group attensira.com serves Kagibot today.
# Kagi's own worked example: crawl everything except# /private/.User-Agent: KagibotAllow: /Disallow: /private/ # Without a Kagibot group, Kagi documents that it follows# your Googlebot rules.# No group names Kagibot, so it reads the wildcard group.User-agent: *Allow: /Disallow: /api/We do not name Kagibot. It falls through to the wildcard group, which allows everything except /api/.
Read the whole file, or write yours with the robots.txt generator.
Who runs Kagibot, and what is it for?
Kagi runs it, and we file it under search indexing. Builds Kagi's search index, so allowing it is how your pages appear in Kagi results. Kagi's crawler page says nothing about its AI features.
From Kagi’s documentation, read . Why a training fetch and a search fetch are different decisions: the AI crawler guide.
Which other crawlers inherit your Googlebot group?
ImagesiftBot documents the same fallback, which makes your oldest robots.txt group the default for crawlers that did not exist when it was written. The case for per-token groups is in the robots.txt chapter.
Why does the casing of Kagibot matter?
Kagi's page heading calls the crawler KagiBot. The user agent and every robots.txt example on the same page write Kagibot. Only the second spelling appears in traffic.
Robots.txt matches the token case-insensitively, so a group written either way works. Log filters and firewall rules often compare case-sensitively, and a filter written from the heading spelling matches nothing, which looks in a dashboard exactly like a crawler that never came. Match kagibot case-insensitively, as a whole word.
How do I verify that a visit really came from Kagibot?
Kagi publishes the four addresses its requests originate from, each paired with a hostname under kagibot.org. The list is typed into a web page, not a machine-readable range file of the kind Google and OpenAI publish, so re-read it when you audit rather than compiling it into your edge config.
The durable check is the round trip: reverse-resolve the accessing address, confirm the hostname ends in kagibot.org, then forward-resolve that hostname and confirm it returns the original address. The full procedure is in verifying a crawler.
- Reverse DNS ends in
- *.kagibot.org
Kagi also documents Published addresses for Kagibot, with no range file to download.
ResearcherTranscribed from Kagi documentation · What user-agent string does Kagibot send?
Match the token, never the version. A rule pinned to a version stops matching silently the next time the operator ships.
The version in this string changes; check Kagi’s page before relying on the exact text.
What does allowing Kagibot cost?
Allowing it costs ordinary crawl load and returns a place in Kagi's results, whose users pay to search. Blocking it removes you from those results and protects nothing Kagi documents using elsewhere.
Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)Should I allow Kagibot in my situation?
Concrete cases, and what each one implies for this agent.
Your firewall flags a burst of requests claiming to be Kagibot from an address you do not recognise.
Check the address against the four Kagi publishes before you allow or block anything.
Kagi publishes the four IP addresses its Kagibot requests come from, each paired with a hostname under kagibot.org. A request from any other address is not Kagi's crawler, whatever its header says, so the check settles the question in one lookup.
You want to be in Kagi but you blocked Googlebot on a section for reasons unrelated to Kagi.
Write an explicit Kagibot group for that section.
The fallback only applies where no Kagibot rule exists, so an explicit group takes precedence over the inherited Googlebot one for this crawler. This is the case the fallback exists to let you handle, and it needs one group rather than a rethink of your Googlebot rules.
A case-sensitive log filter written for KagiBot is returning nothing.
Match kagibot case-insensitively as a whole word.
Kagi writes the crawler's name as KagiBot in the page heading and the token as Kagibot in the user agent and every robots.txt example. Anything comparing case-sensitively against the heading spelling matches none of the real traffic, which reads as an absent crawler rather than a broken filter.
What else do people ask?
What is Kagibot?
Does Kagibot respect robots.txt?
Is Kagibot an AI crawler?
Where do these facts come from?
5 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.
Kagi documents Kagibot as the web crawler for the Kagi search engine.
kagi.com · retrieved
“Kagibot is the web crawler for the Kagi search engine.”
Kagi documents that requests from Kagibot set the User-Agent to "Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)".
kagi.com · retrieved · changes often, check the source
“Mozilla/5.0 (compatible; Kagibot/1.0; +https://kagi.com/bot)”
Kagi states that standard directives in robots.txt that target Kagibot are respected.
kagi.com · retrieved
“Standard directives in robots.txt that target Kagibot are respected.”
Kagi states that if there is no rule targeting kagibot, but there is a rule targeting Googlebot, then Kagibot will follow the Googlebot directives.
kagi.com · retrieved
“If there is no rule targeting kagibot, but there is a rule targeting Googlebot, then Kagibot will follow the Googlebot directives.”
Kagi publishes four IP addresses its Kagibot requests originate from, each paired with a hostname under kagibot.org.
kagi.com · retrieved
“The requests will originate from the following IPs”
Want whatever it takes to win the answer?
7 days free · card required