Chunking

What does a chunk that survives look like?

The passage at the top of this page is one. On 4 October 2026 ChatGPT answered "what is generative engine optimization", and we read the answer verbatim over the Attensira MCP. The sentence about Google had been lifted from a source, cut out, paraphrased into a different document and footnoted, and it still made sense. It carries everything it needs: who says it (Google), what they say (llms.txt, excessive chunking and artificial mentions are not required), and where it applies (Google AI search). Nothing in it says "this", "they" or "as described above".

This is ChatGPT's account of Google's position, quoted from the answer, not from Google's own guide. Read either way, it makes the point this page is about: chunking is the engine's job, not yours. The fix is not to chop your pages into fragments; it is to write sections that still mean something when the engine cuts them out.

Our own pages show the opposite case. Across the 58 tracked questions that do not contain our name, ChatGPT and Perplexity named Attensira in 0 of 4,326 answers in the 30 days to 4 October 2026, read over the Attensira MCP, although they still linked our pages in some of them (the counts are on AI brand mentions). We have not traced every case, but one likely reason is visible on our older pages, which often said "the agent" or "the platform" where the brand name should have been. A retrieval system that cuts such a sentence out keeps the fact and drops the name.

What is chunking?

Chunking is the step where a body of text is cut into pieces small enough to index and retrieve one at a time. It sits at the start of nearly every retrieval-augmented generation system. Anthropic's description of a standard pipeline:

1. break the knowledge base into chunks, usually no more than a few hundred tokens;
2. convert each chunk into a vector with an embedding model;
3. store the vectors in a database that supports vector search;
4. at question time, find the nearest chunks and add them to the model's prompt.

The answer engine never sees your page as a page. It sees whichever chunks won.

Why does chunking lose context?

Because cutting text removes what made it understandable. Anthropic's example: in a collection of financial filings, a relevant chunk might read "The company's revenue grew by 3% over the previous quarter." It names neither the company nor the quarter, so nothing about its vector brings it back for a question that names both, and if it does come back, the model cannot tell what it refers to.

That is what happens to most pronouns, most "this", most section-relative references and most tables whose header row landed in a different chunk.

How do retrieval systems fix it?

Anthropic's fix is contextual retrieval: before embedding each chunk, prepend a short explanation of where it sits in the document, generated by a model and usually 50 to 100 tokens long. The example becomes "This chunk is from an SEC filing on ACME corp's performance in Q2 2023; the previous quarter's revenue was $314 million. The company's revenue grew by 3% over the previous quarter."

Anthropic measured how much that context reduces retrieval failures, alone and paired with BM25 and a reranker; its results are tabled on the BM25 page rather than repeated here.

You cannot count on the engine reading your page doing this. Write as if it does not.

How do you write pages that survive chunking?

• One subject per section. A section covering four topics produces a chunk that means none of them.
• Headings that state or ask the exact thing. The heading is often the only context a chunk keeps.
• Repeat the subject. Write the product name, the version and the date inside the paragraph instead of relying on the heading above it. It is contextual retrieval done by hand.
• Keep identifiers literal. Exact strings are what a keyword index can match when a vector cannot.
• Keep the claim with its qualifier. A caveat in the next paragraph is a caveat in a different chunk.

See AI response optimization for the same rules applied to the sentence an answer quotes.

What are the objections to writing for chunking?

"Chunk size is an engineering setting, not our problem." The size is theirs. Whether a piece of your page means anything on its own is yours.

"Large context windows make chunking obsolete." For small corpora, sometimes: Anthropic says a knowledge base under 200,000 tokens can go in the prompt whole. Web search engines index billions of pages, so they chunk. See context window.

"Repeating the brand name reads badly." Once per section, in the sentence that makes the claim, reads fine to people and is what keeps the name attached when the section travels alone.

How does Attensira handle this?

Attensira reads which of your pages each engine cites and what the answer said, so it can see when a passage was used and your name was dropped. The crew rewrites those passages to stand alone, with your brand and qualifier inside the sentence, and ships the change after your approval, then measures named and linked rates before and after. A passage like the one quoted at the top of this page, which keeps its subject when it travels, is the result we are after for every section you publish. Content optimization shows how those rewrites are proposed and approved.