LLM content optimization

LLM content optimization is the practice of publishing so that a language model can fetch a page, lift a passage from it, and repeat that passage without breaking it. It is three separate problems, access, extraction and correctness under quotation, and they fail independently, which is why work aimed at one does nothing for the others. It is the on-page part of generative engine optimization, the wider discipline that also covers sources you do not own.

Can every AI crawler reach your page?

A model reaches a page through an HTTP request from a named crawler, and each vendor runs several with different jobs. Anthropic's crawler documentation names ClaudeBot (training), Claude-User (fetches a person's question triggers) and Claude-SearchBot (search quality). OpenAI's bot documentation names GPTBot (training), OAI-SearchBot (ChatGPT search), ChatGPT-User (user actions) and OAI-AdsBot (ad landing pages).

"Block the AI crawlers" is therefore not one decision. Refusing a training crawler and refusing a search crawler have opposite effects on whether you can be cited, and a rule aimed at the wrong user agent achieves neither. Changes are not instant either: OpenAI documents that an opt-out from ChatGPT search results takes about 24 hours. See robots.txt and the AI crawler guide.

Access also fails for reasons no directive describes: content that renders only in the browser, a slow response, an interstitial or a login. AI crawlers do not run JavaScript, so a page that needs it is empty to them at any level of writing quality.

Why is the passage, not the page, the unit?

Inside a retrieval-augmented system a page is cut into chunks, the chunks are indexed, and a query selects some of them. The page is never the unit; the section is. See chunking.

That changes almost every writing decision:

• One subject per section. A section covering four topics produces a chunk that represents none of them well.
• Headings that state the question or the claim, not "Considerations".
• Qualifiers inside the sentence they qualify. A caveat two paragraphs up does not travel with the claim.
• Literal identifiers. Version numbers, product names and exact user-agent strings belong in the text, because similarity search is good at paraphrase and poor at exact tokens.
• No dependency on earlier sections. If a passage is only true given what came before, it becomes false when quoted alone.

The test is mechanical: cut any section out and read it cold. If it is still correct and complete, it survives retrieval.

What does "cited but not named" look like?

We have a clean example from our own site. Attensira tracks the question "what is GPTBot" for attensira.com. Between 4 September and 4 October 2026, ChatGPT answered it 33 times. It cited a page on attensira.com in 12 of those answers and named Attensira in none of them; Perplexity answered it 33 times and cited us in none. The answer at the top of this page is one of the twelve: a correct definition, OpenAI's documentation first among its sources, attensira.com eighth, and our name nowhere in the text.

Our page was good enough to be used as a source and still earned no mention, because the passages the model lifted defined GPTBot without saying who wrote them or why it matters to the reader. Citation and mention are different measurements; AI brand mentions covers the difference. The content fix is specific: put the brand and the claim in the same self-contained passage, so whatever is lifted carries the name. That is a rewrite of one section, not a new page.

What can you withhold from AI answers?

Less than you might think. Google's AI features documentation states there are no additional requirements to appear in AI Overviews or AI Mode and that a supporting link must be indexed and snippet-eligible. It names nosnippet, data-nosnippet, max-snippet and noindex as the controls that limit what is shown.

Those controls declare a preference to systems that honour them; they are not enforcement, and they do not exist uniformly across assistants. "Be quoted less" is a weak lever. "Be quoted correctly" is a strong one, and it brings the work back to the passage.

What goes wrong with LLM content optimization?

• Writing for a model's supposed taste instead of for chunk boundaries. There is no documented style bonus.
• Blocking every named agent to protect content, then treating absence from answers as a ranking problem.
• Checking a robots change the same day. One vendor documents roughly a day.
• A page that is correct only as a whole. The most common defect, and invisible to the person who wrote it.
• Treating a citation as success. It records what was retrieved, not that your name or claim survived.

How does Attensira do LLM content optimization?

As work, not as a score. Attensira reads the answers to your tracked questions every day and separates mention from citation, so "cited but not named" shows up as its own gap. The crew then rewrites the passage that should have carried your name, as a pull request against your repository or a draft in your CMS, and it waits for your approval; anything stating a number, price or competitor name waits in every mode. See content optimization. Each change is re-read afterwards and reported per engine with its sample size, including when nothing moved.