Bots
MistralAI-Training: Mistral's dataset bot
MistralAI-Training is Mistral's training crawler. Mistral documents it as collecting web content to build datasets for training its generative models, and says it is not used for search indexing or live answers. Blocking it costs no visibility in Mistral's assistant, because MistralAI-Index and MistralAI-User do that work. Mistral publishes no addresses for it, so visits cannot be verified.
7 days free · card required
0requests filed as MistralAI-Training · 30 days
of 143,601 AI crawler requests from 7 operators
| Agent | AI requests, 30 days | attensira.com robots.txt |
|---|---|---|
| MistralAI-Training (this page) | 0 | Wildcard |
| meta-externalagent | 131,105 | Allowed |
| GPTBot | ≈830 | Allowed |
| ClaudeBot | ≈68 | Allowed |
| Amazonbot | 0 | Allowed |
| Applebot-Extended | 0 | Allowed |
| Google-Extended | 0 | Allowed |
3 of 7 had requests filed in the window. User agents as sent, not checked against address lists.
Analyst · reads the AI traffic boardAI traffic board, read over the Attensira MCP, and our live robots.txt · · n=143,601 AI requests
Karl-Gustav Kallasmaa, Founder & CEOLast updated How do I allow or block MistralAI-Training in robots.txt?
Paste this into the robots.txt at the root of your domain. Beside it, the group attensira.com serves MistralAI-Training today.
# Out of Mistral's training data, in Mistral's answers.User-agent: MistralAI-TrainingDisallow: / User-agent: MistralAI-IndexAllow: /# No group names MistralAI-Training, so it reads the# wildcard group.User-agent: *Allow: /Disallow: /api/We do not name MistralAI-Training. It falls through to the wildcard group, which allows everything except /api/.
Read the whole file, or write yours with the robots.txt generator.
Who runs MistralAI-Training, and what is it for?
Mistral AI runs it, and we file it under training. Crawls web content to help build datasets for training Mistral's generative models. Mistral says it is not used for search indexing or live answers.
From Mistral AI’s documentation, read . Why a training fetch and a search fetch are different decisions: the AI crawler guide.
How do Mistral's three tokens divide the work?
Mistral documents three tokens on one page, each with a statement of what it is not:
- MistralAI-Training: this page. Datasets for model training, not used for search indexing or live answers.
- MistralAI-Index: indexing for Mistral search, documented as not used for generative AI training of any kind.
- MistralAI-User: fetches when a user asks a question, documented as not used to crawl content for training.4
Other operators blur this line. Amazon's training-capable token is a general product crawler that "may" feed training, and Meta's one token covers training or indexing, so the two cannot be separated. OpenAI keeps a clean training and search split across more tokens. Mistral goes one step further by naming each crawler after its job, so the allow-or-block decision needs no interpretation: a robots.txt that blocks -Training and allows the other two says what it means to the next person who opens the file. Our robots.txt generator writes the three as separate groups, and the wider set of blocks is in the robots.txt chapter.
Why not filter logs on the MistralAI- prefix?
All three Mistral tokens share the MistralAI- prefix. A filter on the prefix alone merges a training crawl with a live user fetch, the two jobs this split exists to keep apart. Match MistralAI-Training as a whole word, case-insensitively, and count the other two tokens in their own rows.
How do I verify that a visit really came from MistralAI-Training?
You cannot. Mistral links address files for MistralAI-User and MistralAI-Index, but the training crawler's section lists none, and Mistral publishes no reverse-DNS domain.5 A MistralAI-Training line in your logs is a claim, and AI tokens are the ones scrapers borrow because sites write exceptions for them. Report it as claimed traffic, in a separate column from counts you can prove. The general method is in verifying a crawler.
Mistral AI publishes no reverse-DNS suffix and no IP range file for MistralAI-Training, so a visit can be matched only by its user agent.
ResearcherTranscribed from Mistral AI documentation · What user-agent string does MistralAI-Training send?
Match the token, never the version. A rule pinned to a version stops matching silently the next time the operator ships.
The version in this string changes; check Mistral AI’s page before relying on the exact text.
What does allowing MistralAI-Training cost?
Allowing it means your content may enter Mistral's training datasets. Blocking it costs no visibility in Mistral's assistant, because MistralAI-Index and MistralAI-User do the search and live-answer work under their own tokens.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Training/1.0; +https:/Should I allow MistralAI-Training in my situation?
Concrete cases, and what each one implies for this agent.
You want out of AI training corpora without losing AI referrals.
Disallow MistralAI-Training; leave MistralAI-Index and MistralAI-User allowed.
Mistral documents all three tokens on one page with explicitly different jobs, and states that neither the index crawler nor the user fetcher is used for generative AI training. This is the configuration the three-token split was designed to make possible.
Your robots.txt names MistralAI-User because that is the token you had heard of.
Move the rule to MistralAI-Training if training is the objection.
MistralAI-User is the token behind a fetch triggered by someone asking a question, and Mistral documents it as not used to crawl content for generative AI training. Blocking it stops live answers about you while leaving the training crawler untouched — the exact inversion of what was intended.
You need to report how much of your traffic is AI training crawls.
Count this token, and label the Mistral training row claimed rather than verified.
Mistral publishes address files for two of its three tokens and none for this one. A number you cannot check is still worth reporting; presenting it in the same column as verified counts from OpenAI or Common Crawl is what makes a report untrustworthy.
What else do people ask?
What is MistralAI-Training?
Does blocking MistralAI-Training remove me from Mistral's assistant?
Does it respect robots.txt?
Where do these facts come from?
5 claims on this page, each with the page it was read from; last checked . The numbers in the text point here.
Mistral states that MistralAI-Training crawls web content to help build datasets for training Mistral generative AI models.
docs.mistral.ai · retrieved
“MistralAI-Training crawls web content to help build datasets for training Mistral generative AI models.”
Mistral publishes MistralAI-Training's full user-agent string as "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Training/1.0; +https://docs.mistral.ai/robots)".
docs.mistral.ai · retrieved · changes often, check the source
Mistral states that webmasters can disallow the MistralAI-Training user agent in their robots.txt file, and that Mistral uses specific robots.txt tags to help webmasters manage how their sites and content interact with AI.
docs.mistral.ai · retrieved
“Webmasters can disallow this user agent in their robots.txt file.”
Mistral states that MistralAI-Training is not used for search indexing or to answer live user queries, and separately that MistralAI-Index content is not used for generative AI training of any kind and that MistralAI-User is not used to crawl content for generative AI training.
docs.mistral.ai · retrieved
“This crawler is not used for search indexing or to answer live user queries”
Mistral's crawler page lists published IP addresses for MistralAI-User and MistralAI-Index but lists none for MistralAI-Training, and publishes no reverse-DNS domain for any of its crawlers.
docs.mistral.ai · retrieved
Want whatever it takes to win the answer?
7 days free · card required