# Vector search

> For the complete documentation index, see [llms.txt](https://attensira.com/llms.txt).
> Every page on this site is also served as Markdown: append `.md` to any URL, or send `Accept: text/markdown`.

> Vector search finds text by meaning, comparing embedding vectors instead of words. How it works, a real test with similarity scores, and where it fails.

- Canonical page: https://attensira.com/glossary/vector-search
- Family: glossary
- Last updated: 2026-10-04
- Author: Karl-Gustav Kallasmaa (https://attensira.com/about)
- Claims as data: https://attensira.com/claims.json?slug=vector-search

**In short:** Vector search retrieves text by meaning instead of by shared words. An embedding model turns the question and every passage into lists of numbers, called vectors, and the search returns the passages whose vectors sit closest to the question's. It lets an AI engine find your passage when it shares none of the searcher's words.

**Question vs passage, cosine similarity (bge-small-en-v1.5)**

| Tracked question | Closest test passage | Score | Off-topic passage |
| --- | --- | --- | --- |
| how to block AI crawlers in robots.txt | Disallow GPTBot and OAI-SearchBot in your robots file | 0.840 | 0.517 |
| check if ChatGPT mentions my brand | How to see whether ChatGPT names your company in its answers | 0.871 | 0.426 |
| why is my brand not mentioned by AI | The brand is absent from assistant responses: common causes | 0.767 | 0.498 |

_Source: Attensira embedding test, BAAI/bge-small-en-v1.5 via fastembed 0.8.1, as of 2026-10-04 (3 tracked questions × 5 test passages)._

## Key facts

- Also known as: Dense retrieval; nearest-neighbour search
- How it scores: Cosine similarity between embedding vectors
- Vector size: 384 numbers (bge-small) to 3,072 (text-embedding-3-large)
- Index: [Approximate nearest neighbour, such as HNSW](https://arxiv.org/abs/1603.09320)
- Weak at: Exact names, codes, prices and vague questions
- Related term: [Embeddings, the vectors it compares](/glossary/embeddings)
- What Attensira reads: [Which passages each engine cites, not its index](https://docs.attensira.com/measure/citations-and-sources)

A keyword index asks: which documents contain these words? A vector index asks a different question: which passages mean something close to this? That shift is why an AI engine can answer "how do I keep ChatGPT from training on my site" with a page that only ever says "disallow GPTBot".

## How does vector search work?

Three parts:

1. **An embedding model** turns a piece of text into a vector, a fixed-length list of numbers. Texts with similar meaning get vectors that point in similar directions. The model in our test outputs 384 numbers; OpenAI's text-embedding-3-large outputs 3,072.

1. **An index** stores a vector for every passage. Passages, not pages: documents are split into chunks first, because one vector for a whole page blurs everything on it together.

1. **A similarity measure**, usually cosine similarity, scores the question's vector against the stored ones. The closest passages are returned.

Searching millions of vectors exactly would be slow, so indexes use approximate nearest-neighbour methods such as HNSW, a layered graph that lets a search visit only a small part of the collection.

## What does vector search look like on real text?

We ran a small test on 4 October 2026. Four questions are prompts Attensira tracks for its own brand; the five passages were written for the test. Each was embedded with the open model BAAI/bge-small-en-v1.5 and scored by cosine similarity, where 1 means the same direction. One passage, "Best project management software for agencies", was deliberately off-topic.

Two things show up. First, the matches the model got right share almost no words with the question. "Block AI crawlers in robots.txt" and "disallow GPTBot and OAI-SearchBot in your robots file" have only "in" and "robots" in common, and still scored 0.840 against 0.517 for the off-topic passage. Keyword search would have struggled; vector search did not.

Second, the vague question failed: its five passages scored between 0.576 and 0.628, a spread of 0.052, and the off-topic passage scored 0.580. When the question carries little meaning, the scores go flat, and an off-topic passage can rank beside a relevant one. Real engines handle this by rewriting vague questions into several specific searches and by combining vector results with keyword results before [reranking](/glossary/reranking).

## What happens when the question carries little meaning?

The scores go flat. The fourth tracked question in the same test, "how to get recommended by AI", scored almost the same against all five passages, and the off-topic one about project management software came out level with a relevant one. Engines respond by rewriting vague questions into several specific searches and adding keyword results.

| Figure | What it counts | n |
| --- | --- | --- |
| 0.628 | closest passage: an OpenAI Help Center title about searching with ChatGPT | best of 5 passages |
| 0.580 | the off-topic passage, "Best project management software for agencies" | 1 of 5 passages |
| 0.576 | the lowest-scoring passage | worst of 5 passages |
| 0.052 | the whole spread from best to worst | 5 passages |

Window 2026-10-04 to 2026-10-04. Attensira embedding test, BAAI/bge-small-en-v1.5 via fastembed 0.8.1, cosine similarity, run 2026-10-04.

_Source: Attensira embedding test, BAAI/bge-small-en-v1.5 via fastembed 0.8.1, as of 2026-10-04 (1 tracked question × 5 test passages)._

## Why does it decide whether your page gets read?

Because in most AI engines it is one of the first filters. If your passage is not near the question in vector space, it never reaches the reranker or the model, however good it is. Two consequences for writing:

- **Say what the passage is about in plain terms.** A passage that names its subject and answers one question embeds close to that question.

- **Keep the exact words too.** Names, product terms and numbers are where vectors are weakest and keyword search is strongest. A page that uses both the buyer's phrasing and the precise term is found by both.

## What do people get wrong about it?

- **Thinking keywords no longer matter.** Engines combine both kinds of search. Exact terms still decide the exact-term questions.

- **Embedding whole pages.** A page about five things produces a vector that is close to none of them.

- **Reading a similarity score as a probability.** 0.84 means "closer than the others", not "a probability of relevance". Scores are only comparable within one model.

- **Assuming one engine's retrieval is everyone's.** Each engine uses its own models, chunking and indexes.

## How does Attensira use this?

Attensira cannot see inside any engine's vector index. It measures the result: which passages each engine cites for each question, every day. When a page that should answer a question is never cited, the crew checks the passage against the question, the wording buyers use and the exact terms, and drafts the rewrite for your approval. The free [answer passage extractor](/tools/answer-passage-extractor) shows which passages on one of your pages have the shape an engine can lift whole.

## FAQ

### How is vector search different from keyword search?

Keyword search, such as BM25, scores documents by the words they share with the query. Vector search compares embeddings, numeric representations of meaning, so a passage about disallowing GPTBot can match a question about blocking AI crawlers with almost no words in common. Most production systems run both and merge the results.

### What distance measure does vector search use?

Usually cosine similarity, the angle between two vectors, or dot product on vectors normalised to length one, which gives the same ranking. Some systems use Euclidean distance. The embedding model's documentation says which it was trained for.

### How many dimensions does an embedding have?

It depends on the model. The small open model used in the test on this page produces 384 numbers per text; OpenAI documents 1,536 for text-embedding-3-small and 3,072 for text-embedding-3-large. More dimensions can hold finer distinctions and cost more to store and search.

### How does vector search stay fast over millions of passages?

With an approximate nearest-neighbour index, such as HNSW, which builds a layered graph of vectors so a search visits a small fraction of them. It trades a little accuracy for a large gain in speed.

### What is vector search bad at?

Exact things: product names, model numbers, error codes, prices and short or vague queries. Meaning-based scores for a vague question come out nearly flat, so an off-topic passage can rank beside a relevant one. That is why engines combine it with keyword search and rerank the results.