Page token inspector

The cut line is a number you choose, not one we measured. No answer engine publishes how much of a page it keeps, and the real figure moves with the product, the model and the query. We start you at 2,000 tokens as a reference point; change it to whatever budget you want to test against.

Give it a page and it extracts the visible text, splits it on headings, and estimates the tokens each section costs at about 4 characters per token. Then it draws the line where a retriever keeping only the first N tokens would stop reading, and tells you which sections sit below it.

The one thing it usually shows first: how far down the page your own content starts, once the nav, the banner and the cookie notice have taken their share.

How the token count is produced

The page is fetched once, server-side. Everything inside script, style, noscript, svg, template, head, iframe and canvas is discarded, HTML entities are decoded, and runs of whitespace are collapsed. What survives is the visible text, in document order.

That text is split at every h1 to h6. Everything before the first heading becomes the preamble — on most sites that is the announcement bar, the navigation and the cookie notice, whether or not any of it is inside a nav element. Text that is inside nav, header, footer or aside is counted separately so you can see how much of the budget is spent on furniture that repeats on every page.

Tokens are then estimated at about 4 characters per token, applied to each block of text and summed. That ratio is the common rule of thumb for English prose in byte-pair vocabularies. It is not a measurement of any specific tokenizer, and this tool does not run one.

So these are approximations, and we will not dress them up as anything else. A real tokenizer would give you a precise number for one model and the wrong number for the next. What does not change between tokenizers is the shape: which sections are expensive, how much of the page is furniture, and how far down your answer sits. That shape is what this tool is for. Where the ratio is least fair — text outside the Latin alphabet, which it under-counts — the result says so instead of quietly reporting a low number.

What the cut line is, and what it is not

The cut line is the number you typed. It is a chosen reference point, not a measured constant, and it is worth being blunt about why: no answer engine publishes how many tokens of a fetched page it keeps before answering. Retrieval budgets differ by product, by model, by query and by how many other documents are competing for the same context. Anyone quoting you a single authoritative number for that has invented it.

What the tool does with your number is arithmetic, and the arithmetic is honest: it walks the sections in document order, accumulates the estimated tokens, and reports where your number falls. Move the line and everything moves with it. Testing the same page at 800, 2,000 and 8,000 tokens tells you how sensitive the page is to a budget you cannot control — which is more useful than pretending to know the budget.

It also does not know which sentence answers your reader. It reports where the page's content starts and which sections fall below the line. Whether the section that got cut was the one that mattered is a judgement only you can make, so the tool declines to make it for you.

What to do when your answer is below the line

Move the answer up, before you cut anything. A page that states its answer in the first paragraph survives every budget. The most common fix here is not deletion, it is order.
Count what the preamble costs. A mega-menu, an announcement bar and a cookie notice can eat several hundred tokens before the first heading. They are usually the same on every page, so every page pays for them.
Put the furniture in nav, header, footer and aside. Semantic regions are how anything downstream can tell your boilerplate from your body copy. This tool uses exactly that signal, and so does most extraction code.
Break long sections with real headings. A 4,000-token section is one indivisible lump: a retriever that cannot fit it either takes an arbitrary prefix or drops it. Headings give it seams to cut along.
Check the section the cut lands inside. A section split in half is the one to look at first, because half of it is being read out of context.

Why the token budget matters for being cited

A model does not read your page the way a person does. Something fetches it, strips it, and hands a bounded slice of the text to the model along with the question. What is in that slice is what can be quoted, summarised or cited. What is outside it may as well not exist for that answer.

That is why the ordering of a page turns out to matter more than its length. A 12,000-token page whose answer is in the first 300 tokens is in better shape than a 2,000-token page that spends 900 tokens on navigation before it says anything. Length is not the problem; spending the beginning of the budget on furniture is.

Questions people ask about token budgets

Is this the same count I get from tiktoken?

No, and it does not try to be. This is a characters-per-token estimate at a ratio of about 4:1, stated on every number it produces. A real byte-pair tokenizer would be exact for one vocabulary and wrong for the next one, and there is no single vocabulary in use across ChatGPT, Claude, Gemini and Perplexity. Expect the right order of magnitude and reliable ratios between sections, not a figure to quote to three digits.

Where does the default cut of 2,000 tokens come from?

From us, as a starting point, and nowhere else. It is not a measured property of any assistant and we are not going to imply otherwise. It exists so the line has somewhere to sit before you move it. If you have your own evidence about the budget your pages face, type that number in instead.

Do you see what ChatGPT or Claude fetched from me?

No. Nothing here observes traffic from any assistant. This tool fetches your page once, itself, because you asked it to. The only record of what crawlers actually requested from your site is your own server logs, which is a different question with a different answer.

The report says my page has almost no tokens. Is it broken?

Probably not. It usually means the page builds its body in the browser with JavaScript, which we do not execute. That is worth knowing rather than working around: a crawler taking the raw HTML sees the same near-empty document. If the content matters for being cited, it needs to be in the HTML that is served.

Should I just make my pages shorter?

Not on this evidence alone. Length is not what this measures — position is. A long page whose answer sits near the top is fine. Cutting a page in half without moving the answer up changes the total and not the problem. Look at where the first heading starts and what falls below the line before deleting anything.

Do you store the pages I inspect?

No. The page is fetched, counted inside the request, and the result is returned to your browser. There is no account, no email field and no saved history.