BERT

Why do small words change the answer?

The table above holds two questions we track for attensira.com. A buyer who types either one wants the same thing: to know why an AI assistant does not recommend their company. The engines did not treat them as the same question.

"Why is my brand not mentioned by AI" got answers about brand visibility. ChatGPT drew on a spread of sources, Search Engine Land and Google's developer documentation among them, and across both engines the most-named brand was Semrush, in 7 of 115 answers. "Why am I not showing up in ChatGPT" went somewhere else. ChatGPT's stored answer from 3 October opened with ChatGPT profiles, which are private by default, and reached website visibility only as the second reading. Over 30 days, 57% of the citations ChatGPT attached to that question came from OpenAI's help centre, against 8% on the other phrasing.

The difference is two words. "My brand" names the thing; "I" and "showing up" let the engine read a person and a profile. These engines are not running Google's BERT, but this is the idea BERT brought to Search in 2019: meaning lives in the small words and the order, not only in the keywords. Your buyers phrase one need several ways, each phrasing is a separate question an engine may answer differently, and a page written around a keyword answers only the phrasings that happen to share its reading.

What is BERT?

BERT stands for Bidirectional Encoder Representations from Transformers. The paper by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova was submitted on 11 October 2018 and set new results on standard language benchmarks, including a GLUE score of 80.5% and SQuAD v1.1 test F1 of 93.2.

"Bidirectional" is the important word. BERT is pre-trained by hiding words and predicting them from the words on both sides, so every word's representation depends on its whole context, not only on what came before. That is what lets a preposition in the middle of a query change the meaning of the words around it.

What did BERT change in Google Search?

Google announced on 25 October 2019 that BERT would help Search better understand one in ten searches in the US in English, and that it was also applied to featured snippets. The same post noted that 15 percent of the queries Google sees each day are new to it, which is why understanding a query's wording matters more than matching its keywords.

Google's own example was a query about travelling to the US from Brazil, where the word "to" decides who needs the visa. Before BERT, results drifted toward the opposite direction of travel. After, they matched the direction the person meant.

Google has not published how much weight BERT carries today; the 2019 announcement is the last figure it gave.

How is BERT related to today's AI search?

BERT is an encoder: it reads text and produces representations of meaning. The models behind ChatGPT, Claude and Gemini are much larger and generate text, but they descend from the same transformer architecture. The embeddings that retrieval systems use to match a question to a passage are produced by encoder models in BERT's family. So the habit BERT rewarded, writing sentences that state relationships plainly, is the same habit that makes a passage easy for an AI engine to retrieve and quote. See semantic search.

What are the common misreadings of BERT?

"Optimise for BERT." There is nothing to optimise for. No markup, no keyword density, no special format. Google said as much at launch.

"BERT replaced keywords." It changed how queries are read. Exact identifiers, names and numbers still need to be on the page as text; lexical matching still matters, which is why systems pair meaning-based retrieval with methods such as BM25.

"BERT is how ChatGPT ranks pages." It is not. Each AI engine has its own retrieval and ranking. What carries over is the principle, not the system.

"Long-tail pages for every phrasing." Two questions with the same keywords and different intent need different answers, not two near-duplicate pages. See search intent.

How does Attensira use this?

Attensira tracks the questions your buyers actually ask, in their words, and reads how each engine answers each phrasing separately. When two phrasings of your topic get different answers, the crew writes or fixes the page that owns each one so the passage answers that exact question, ships the change after your approval, and measures each phrasing on its own. Prompt tracking shows how phrasings are tracked side by side.