Bots
MistralAI-Index: the search crawler
MistralAI-Index is Mistral's search-indexing crawler. It indexes pages for Mistral search, which answers questions in Mistral's assistant, and Mistral states the content is not used for generative AI training of any kind. Blocking it costs you visibility in Mistral's answers and protects nothing. Mistral publishes its addresses at mistral.ai/mistralai-index-ips.json, so visits can be verified.
7 days free · card required
0requests filed as MistralAI-Index · 30 days
of 143,601 AI crawler requests from 7 operators
| Agent | AI requests, 30 days | attensira.com robots.txt |
|---|---|---|
| MistralAI-Index (this page) | 0 | Wildcard |
| OAI-SearchBot | ≈6,480 | Allowed |
| PerplexityBot | 621 | Allowed |
| Amzn-SearchBot | 0 | Parent's rule |
| Claude-SearchBot | 0 | Allowed |
| meta-webindexer | 0 | Wildcard |
| YouBot | 0 | Allowed |
2 of 7 had requests filed in the window. User agents as sent, not checked against address lists.
Analyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP, and our live robots.txt · · n=143,601 AI requests
Karl-Gustav Kallasmaa, Founder & CEOLast updated How do I allow or block MistralAI-Index in robots.txt?
Paste this into the robots.txt at the root of your domain. Beside it, the group attensira.com serves MistralAI-Index today.
# Be findable in Mistral's answers.User-agent: MistralAI-IndexAllow: /# No group names MistralAI-Index, so it reads the wildcard# group.User-agent: *Allow: /Disallow: /api/We do not name MistralAI-Index. It falls through to the wildcard group, which allows everything except /api/.
Read the whole file, or write yours with the robots.txt generator.
Who runs MistralAI-Index, and what is it for?
Mistral AI runs it, and we file it under search indexing. Indexes pages for Mistral search, which answers questions in Mistral's assistant. Mistral states this content is not used for generative AI training of any kind.
From Mistral AI’s documentation, read . Why a training fetch and a search fetch are different decisions: the AI crawler guide.
How does MistralAI-Index differ from MistralAI-Training?
They share a prefix and nothing else that matters to the decision. MistralAI-Index fetches for Mistral search. MistralAI-Training fetches to build training datasets, and Mistral documents it as not used for search indexing or live answers.2 Mistral publishes an address file for the index crawler and none for the training one.
So if you disallow one Mistral token, disallow the training one. The usual configuration is an allow group for MistralAI-Index and a disallow group for MistralAI-Training, each named exactly. The third token, MistralAI-User, fetches live when a person asks a question. The trade-off is drawn out in training crawlers versus search crawlers.
Why write an Allow group when nothing blocks MistralAI-Index?
An explicit Allow group for a token you want survives the next person who adds a restrictive wildcard group in a hurry, because a crawler with its own group ignores the wildcard. And because Mistral's three tokens share a prefix, address them individually: a group for MistralAI is not a wildcard and matches no token at all.
The full set of blocks is in the robots.txt chapter, and our robots.txt generator writes the three-token Mistral configuration.
How do I verify that a visit really came from MistralAI-Index?
Mistral publishes this crawler's addresses at mistral.ai/mistralai-index-ips.json, with a separate file for MistralAI-User. Both were reachable on 3 September 2026.
Check the source address of any request carrying the token against the file. One from an address outside it is not Mistral, whatever the header says. Because each crawler has its own file, a verified address also tells you which Mistral token called. MistralAI-Training has no file, so index traffic can be verified and training traffic cannot. The full procedure is in verifying a crawler.
- IP range file
- mistralai-
index- ips.json
Mistral AI publishes no reverse-DNS suffix for MistralAI-Index; verify by the range file.
ResearcherTranscribed from Mistral AI documentation · What user-agent string does MistralAI-Index send?
Match the token, never the version. A rule pinned to a version stops matching silently the next time the operator ships.
The version in this string changes; check Mistral AI’s page before relying on the exact text.
What does allowing MistralAI-Index cost?
Allowing it costs crawl load and puts your pages in the index Mistral's assistant searches. Mistral states the content is not used for generative AI training of any kind, so blocking it protects nothing and costs your visibility in Mistral's answers.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Index/1.0; +https:/Should I allow MistralAI-Index in my situation?
Concrete cases, and what each one implies for this agent.
A wildcard AI blocklist swept up every token containing "MistralAI".
Allow MistralAI-Index explicitly and keep the disallow on MistralAI-Training only.
Mistral states this crawler indexes for search and that its content is not used for generative AI training of any kind. A prefix-based block trades away the indexing path that puts your page in an answer, in exchange for nothing that the training rule was not already doing.
You allowed the token but see no MistralAI-Index requests at all.
Check the edge before the robots.txt file — allowlist the addresses in mistralai-index-ips.json.
A permissive robots.txt behind a WAF that refuses the connection is invisible to the operator; the crawler never gets far enough to read your permission. This is the most common cause of a site being absent from an index it believes it allowed.
You want to know whether Mistral is indexing you or training on you.
Split the tokens and verify the index requests against the published file.
Mistral documents three tokens with different jobs and publishes address files for two of them. The index file lets you turn claimed index traffic into verified index traffic, which is the only number that supports a statement about whether you are actually being crawled for Mistral search.
What else do people ask?
What is MistralAI-Index?
How is this different from MistralAI-User?
Does it obey robots.txt?
Where do these facts come from?
5 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.
Mistral states that MistralAI-Index is for automated crawling of the web for indexing purposes only, and that it indexes content for Mistral search, which helps answer user questions in Mistral's assistant.
docs.mistral.ai · retrieved
“MistralAI-Index is for automated crawling of the web for indexing purposes only.”
Mistral states that content crawled by MistralAI-Index is not used for generative AI training of any kind, and documents MistralAI-Training separately as the crawler that builds datasets for training Mistral generative AI models.
docs.mistral.ai · retrieved
“Content crawled by MistralAI-Index is not used for generative AI training of any kind.”
Mistral publishes MistralAI-Index's full user-agent string as "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Index/1.0; +https://docs.mistral.ai/robots)".
docs.mistral.ai · retrieved · changes often, check the source
Mistral states that it uses specific robots.txt tags to help webmasters manage how their sites and content interact with AI, and documents MistralAI-Index as one of those user agents.
docs.mistral.ai · retrieved
“To help webmasters manage how their sites and content interact with AI, Mistral uses specific robots.txt tags.”
Mistral publishes the addresses MistralAI-Index fetches from at mistral.ai/mistralai-index-ips.json, and a separate file for MistralAI-User.
docs.mistral.ai · retrieved
Want whatever it takes to win the answer?
7 days free · card required