Is llms.txt worth publishing?

What happened when we published an llms.txt?

One engine read it, and cited it. Attensira publishes an llms.txt file and a longer llms-full.txt at the root of attensira.com. When Perplexity answered questions about us by name on 2 and 3 October 2026, some of its answers listed attensira.com/llms-full.txt among their sources, next to our home page, our about page and our documentation. The count is at the top of this page; we read every one of those answers verbatim over the Attensira MCP.

So the file is not invisible. At least one engine fetched it, read it and treated it as a source about us.

Three things temper that.

It was only for our branded questions. Perplexity cited the file when asked about Attensira by name, and ChatGPT never cited it. Neither engine cited it on the unbranded category questions we track, where we were named in no answers in the month.

It did not fix what the answers said. The same answers described us partly in older positioning lifted from other pages. Having a clean summary available did not stop the engine using older sentences elsewhere.

Ours is too big. Size matters for a file whose purpose is to fit in a model's context, and the proposal's whole idea is a concise file. When we measured ours on 4 October 2026 it had grown into an index of the site; the sizes are further down this page.

That is the honest summary of llms.txt as of October 2026: cheap, read by some agents, cited by at least one, and not documented by any major operator as an input to what it crawls or cites.

What is llms.txt?

A proposal, not a standard. Jeremy Howard published it on 3 September 2024 at llmstxt.org, "to standardise on using an /llms.txt file to provide information to help agents use a website".

The motivation is practical. A web page built for people carries navigation, scripts, banners and layout that an agent has to strip away to find the content, and a whole site rarely fits in a model's context. A short Markdown file at a known address, written for agents, solves both: it says what the site is and points at the pages worth reading, in plain text.

It is not a crawler control. The proposal says it is meant to coexist with robots.txt and sitemaps. robots.txt says who may fetch what. A sitemap lists every page. llms.txt is a curated reading list.

What goes in the file?

The proposal defines a small, fixed shape, in this order:

1. An H1 with the site's name. The only required element.
2. A blockquote with a short summary, the key facts an agent needs to understand everything after it.
3. Optional paragraphs or lists with more detail.
4. H2 sections of links, each link a Markdown link with an optional note after a colon.
5. An "Optional" section for secondary links an agent can skip when its context is tight.

A minimal file for a fictional company:

# Ledgerly

> Ledgerly is invoicing software for agencies of 5 to 50 people. It sends
> invoices from tracked time, chases late payments and syncs to Xero.

## Product
- [Pricing](https://ledgerly.example/pricing.md): plans and what each includes
- [How it works](https://ledgerly.example/how-it-works.md): from time entry to paid invoice

## Docs
- [Xero integration](https://ledgerly.example/docs/xero.md): setup and sync rules

## Optional
- [Changelog](https://ledgerly.example/changelog.md)

The proposal also suggests offering clean Markdown versions of pages at the same URL with .md added, so the links can point at text rather than HTML. Attensira serves one for every page; attensira.com/pricing.md returns the pricing page as Markdown.

Do AI crawlers read llms.txt?

Some agents fetch it. No major operator documents using it.

None of the crawler documentation from OpenAI, Anthropic, Perplexity, Google, Apple, Amazon or Meta that we reviewed for this guide says the operator reads llms.txt or uses it to decide what to crawl, index or cite. Google goes further: it says you do not need to create new machine-readable files, AI text files or markup to appear in its AI features.

That leaves direct evidence, and the evidence is thin and mixed. Perplexity citing our llms-full.txt shows that at least one engine's retrieval can surface the file as a source. It does not show the file changed what was said. Documentation publishers such as Mistral serve one, so agents working with their docs have a clean entry point.

So: treat it as a convenience for agents that look for it, not as a lever.

Is it worth publishing one?

Yes, for most sites, as long as you are clear about why.

It is cheap. If your pages come from structured content, generating the file is a few lines of code. If not, a hand-written file of twenty links takes an hour.

It is harmless. It does not block, redirect or change anything else.

It helps the agents that use it. Coding assistants and research agents that look for it get a clean summary instead of scraping your navigation.

It forces a useful exercise. Writing the blockquote, two or three sentences that say what you are and for whom, is the same exercise as writing the passage you want AI answers to quote. If you cannot write it, your home page probably does not say it either.

What it is not worth is time taken from the work that does move answers: letting search agents in, writing passages that stand alone, and earning a place on the third-party pages that decide your category.

How should you write a good one?

Five rules, most of which we broke with our own file.

1. Lead with a blockquote that would make a good answer. Name, category, who it is for, the one thing that sets you apart. This is the part most likely to be quoted.
2. Keep it short. Tens of links, not hundreds. Put everything secondary under "Optional".
3. Link to Markdown where you can. The point is to spare the agent the HTML.
4. Keep it in sync with the site. Generate it from the same source as your pages, or review it whenever positioning changes. A stale llms.txt is a stale passage at a well-known address.
5. Do not put anything in it you would not put on the page. Content written only for bots, or instructions aimed at models, is a risk with no documented benefit.

The llms.txt generator drafts a file in the proposal's format from your site.

What does a good summary blockquote look like?

Ours is a fair example of the content and a poor example of the length. It opens: "Attensira is an AI-native SEO agency run by software: the service of an SEO agency at the price of a tool." It then says who the crew is, which engines it reads, what kinds of work it ships, who it is for, and how approval works.

Everything in it is a sentence we would be glad to see quoted, and each sentence names us, so it survives being lifted alone. Its weakness is that it runs to well over a hundred words before the first link. A tighter version would keep the first sentence, add one sentence on what the crew does and one on who it is for, and move the rest into the linked pages. That is the shape to aim for: three sentences an engine could use as the whole answer to "what is this company?".

Write yours the same way you would write the answer block at the top of your home page: plain words, your name in every sentence, no claims you cannot back. If the two disagree, fix whichever is out of date.

How can you tell whether agents read your file?

Look in two places.

Your server or edge logs. Filter requests for /llms.txt and any companion files, and group them by user agent. Agents that request it by name are agents that look for it. Remember that the user agent is self-reported; the verification chapter shows how to check it.

The answers. Track a branded question such as "what is [your company]" on each engine and read which sources the answers attach. That is how we saw Perplexity citing our llms-full.txt. If the file never appears as a source and nothing in the answers echoes its summary, it is not doing much, and that is fine for a file that costs so little.

What neither place can tell you is whether the file changed an answer. For that you would need a before-and-after on the same questions, with the file as the only change, which almost nobody has run, ourselves included. If you run one, publish the result either way.

What should you do instead, or as well?

If the goal is to be named and cited in AI answers, three pieces of work matter more than any file:

• Allow the search and user agents of the engines you care about in robots.txt. The robots.txt chapter shows how.
• Serve your answers in the HTML, readable without JavaScript, with passages that name you and stand alone.
• Be accurately present on the third-party pages engines already cite for your buyers' questions.

Publish an llms.txt alongside that work. Just do not count it as the work.

The last chapter of this guide, what to allow and what to block, brings the access decisions together.