AI content detection

AI content detection is the attempt to decide, from a piece of text, an image or an audio file alone, whether it was produced by a generative model. It is a classification problem with no ground truth in the artefact, which is why it behaves so differently from the way people expect a detector to behave.

Two families of technique get called by the same name, and only one of them is detection in the strict sense.

Post-hoc classifiers

A post-hoc classifier reads the finished artefact and outputs a likelihood. For text, the features are statistical: how predictable each token is given the ones before it, how little the predictability varies across the passage, how narrow the vocabulary is. Machine text tends to sit closer to the high-probability path than human text wanders.

The failure mode follows directly from what is being measured. The classifier reads typicality, not authorship, so anything that makes human writing more typical pushes it toward the machine class:

• Edited, house-style, formulaic prose — press releases, legal boilerplate, exam answers.
• Writing by a non-native speaker drawing on a smaller construction set.
• Short samples, where there is simply not enough text for the distributions to separate.

And the reverse holds. A machine draft that a person has rewritten, reordered, or given an idiosyncratic sentence to carries the human class again. Because the two error types trade against each other on the same threshold, a detector tuned to catch more machine text falsely accuses more people, and there is no setting where both are small. That is why an individual accusation — a student, a job applicant, a contributor — cannot rest on a classifier score alone.

Provenance and watermarking

The second family does not detect anything; it labels at the point of creation.

Provenance signing attaches cryptographically signed metadata to an asset as it is made or edited. The C2PA specification, version 2.2, published May 2025, defines a manifest that binds assertions about how an asset was created and changed to a claim signature, in a tamper-evident structure. Its scope is deliberately narrow: the spec states that C2PA specifications should not provide value judgments about whether a given set of provenance data is good or bad, only whether the assertions can be validated as associated with the asset, correctly formed and free from tampering. It tells you what a signer claimed, and that nobody altered the claim. It does not tell you the claim is true, and stripped or absent metadata is the normal case, not a red flag.

Watermarking embeds a signal in the output itself. Google DeepMind describes SynthID as embedding digital watermarks directly into AI-generated images, audio, text or video — imperceptible to humans, and for images and video designed to survive cropping, filters, frame-rate changes and lossy compression. This is far more robust than reading token statistics, with one structural limit: it only exists where the generator chose to put it. A model that does not implement the scheme emits nothing to find, so a negative result is uninformative.

Both approaches share that asymmetry. A present, valid signal is strong evidence. An absent one is no evidence at all.

Why this matters for being cited by an agent

Publishers usually arrive at this term from one anxiety: will AI-assisted pages be demoted or excluded?

Google's answer is about purpose rather than provenance. Its spam policies define scaled content abuse as many pages generated for the primary purpose of manipulating search rankings and not helping users, and give generative AI tools producing many pages without adding value as an example of how that happens. Its helpful-content guidance separately suggests adding automation and AI disclosures where they would reasonably be expected. Neither says machine-assisted writing is disallowed; both say output produced at volume without added value is.

The practical reading, for a page you want an agent to fetch, quote and attribute: production method is not the lever. Whether the page contains a fact worth extracting is. A page carrying primary sources with dates, a named author, a real last-reviewed date and a claim that is checkable is useful regardless of what drafted it — and no detector applied to the prose changes that. See GEO, crawling and indexing and AI training data.

Failure modes to avoid

1. Acting on a percentage. A score is a likelihood over a distribution, not a finding about one document. Institutional decisions taken on it will be wrong at a rate the vendor rarely publishes.
2. Reading absence as proof. No watermark and no manifest is the default state of most content on the web.
3. Confusing provenance with truth. A signed asset can assert something false; the signature covers integrity, not accuracy.
4. Rewriting to beat a detector. It optimises for statistical atypicality, which is orthogonal to whether the passage answers a question well.
5. Publishing volume because generation got cheap. That is precisely the behaviour the scaled-content policy names.

Frequently asked questions

Is AI-written content penalised?

Not for being AI-written. The policy targets pages made at scale primarily to manipulate rankings without adding value.

Why do detectors flag human writing?

They measure statistical typicality, not authorship, so formulaic and non-native writing is over-flagged.

What does C2PA prove?

That the signed assertions are intact and associated with the asset — not that they are true.

Does a missing watermark mean human-made?

No. Only cooperating generators embed one, so absence carries no information.