llms.txt

llms.txt is a proposed convention for publishing a short Markdown file at /llms.txt that gives an agent a curated, expert-level map of a site: what the project is, and where the detail lives. It was proposed by Jeremy Howard in a document published on 3 September 2024, now in a v2 revision dated 10 August 2026.

The argument behind it is about cost, not permission. Web pages are built for people: HTML wraps information in navigation, scripts and layout, and turning that back into clean text is hard and imprecise. Context windows are larger than they were and still too small for most sites, and every wasted token costs time and money. So the proposal asks a site to do the extraction once, deliberately, instead of every agent doing it badly.

What goes in an llms.txt file?

The spec fixes an order, so the file can be read by a parser as well as a model:

• an optional byte-order mark;
• an H1 with the name of the project or site, the only required section;
• a blockquote with a short summary of the key information;
• zero or more Markdown sections, any type except headings, with more detail;
• zero or more H2-delimited sections, each a list of links in the form [name](url), optionally followed by a colon and notes.

By convention a section headed "Optional" holds links an agent can skip when it needs a shorter context. The file stays small; the detail lives behind the links.

What does the second half of the proposal ask for?

llms.txt is usually discussed as a single file. The proposal asks for two things, and the second does more work: that pages an agent might need also offer a clean Markdown version at the same URL, with .md appended or the extension replaced.

To make those discoverable it recommends standard link relations: rel="alternate" type="text/markdown" pointing at the Markdown version, and rel="describedby" pointing at the llms.txt that covers the page, as <link> elements or an HTTP Link: header.

What does a working llms.txt look like?

You are reading a site that publishes one, so here is ours as it stood on 4 October 2026 rather than a template. attensira.com/llms.txt opens with the H1 # Attensira and a two-paragraph blockquote saying what the company is. It then runs through H2 sections such as "What Attensira Does", "Core Capabilities", "Glossary", "Guides" and "AI Crawlers and Bots", with each page listed beside a one-line description. We do not quote its length: it changes whenever pages are added or retired, which is the point of generating it.

Three things about it are worth copying:

1. It is generated, not hand-kept. The capability list comes from the same registry the feature pages are built from, so the file cannot name a capability the site does not have.
2. Every page has a Markdown twin. Append .md to any URL on this site, prefix the path with /md, or send Accept: text/markdown, and you get the same page as plain Markdown with no navigation; for this glossary, each claim's source URL and retrieval date come with it.
3. It is kept complete. The file is built from the same page lists the sitemap is built from, so a new page appears in both or in neither, and every page it links to has a Markdown twin.

What it does not do is guarantee anything, and our own tracking says so. One of the questions we track is which AI visibility tool to use for docs built on Mintlify or Docusaurus. In the answer at the top of this page ChatGPT treats llms.txt as one item of "technical readiness", next to Markdown endpoints and crawler access, and credits Mintlify with generating it. Over 30 days to 4 October 2026, mintlify.com took 239 of the 484 citations in ChatGPT's answers to that question, and none of the 24 answers on either engine named Attensira, llms.txt and all. Publishing the file put us on the map an agent reads; it did not put us in the answer.

How does llms.txt relate to robots.txt and sitemaps?

It is designed to sit beside them. robots.txt is about access and is consulted before a fetch; llms.txt is consulted on demand by an agent already allowed to read you. XML sitemaps list indexable pages for search engines; the proposal notes they usually will not list Markdown versions, cannot include useful external URLs, and are too large in aggregate for a context window.

The proposal also considered /.well-known/ and rejected it: a well-known URI exists only at the origin root, and many authors control only a path on a shared host. An llms.txt can describe the path it sits in.

What are the limits of llms.txt?

Two worth holding onto.

It is downstream of access. An agent disallowed in robots.txt, or blocked at your CDN, never reaches your llms.txt. Fix access before discovery.

It is a proposal, not a standard. There is no ranking authority behind it. The real benefit is narrower: agents that follow it spend fewer tokens finding the right page and land on prose you wrote for them. The v2 revision says documentation is where the files are used most, with coding agents following them to API references and tutorials.

Does Attensira help with llms.txt?

Partly, and we keep the line clear. The free llms.txt generator builds a file from your sitemap and validates an existing one line by line, and the AI crawler guide covers where it fits among the files crawlers read. The paid product does not maintain an llms.txt for you. Its crew works on the pages the file points at: finding the questions where answers cite someone else, rewriting the passage that should have been quoted, and shipping it as a pull request or CMS draft after your approval. The useful order of work is the reverse of the file's: get the pages it would list into answers first, and write the index of them second.