Vector search
Retrieval by geometric proximity between embedding vectors rather than by word overlap, and the reason a passage about your product can be found without containing the words that were searched.
Karl-Gustav Kallasmaa, Founder & CEOLast updated Vector search is retrieval by geometric proximity: content and query are each converted into a vector by an embedding model, and the system returns the stored vectors nearest the query vector under some distance metric. Nothing about the query's words is matched. What is matched is a position in a space where the model has learned to place things that mean similar things close together.
This is the retrieval half of most systems that answer questions from documents, including RAG pipelines. Understanding it matters to a publisher for one reason: it decides which span of your writing is eligible to be read at all.
The mechanism
Four steps, each with its own settings.
Chunking. A corpus is cut into spans — a section, a paragraph, a sliding window. One vector is computed per span, and that vector is the only thing retrieval sees. The span is therefore the real unit of the index, not the page.
Embedding. Each span goes through a model that emits a fixed-length array of floats. OpenAI documents text-embedding-3-large at 3072 default dimensions and text-embedding-3-small at 1536, both shortenable by passing a dimensions parameter, which trades recall for storage and speed.
Indexing. Comparing a query against every stored vector is linear in corpus size, so the vectors go into an approximate-nearest-neighbour index. One widely implemented index is HNSW, described by Yu. A. Malkov and D. A. Yashunin in a paper first submitted in March 2016 and last revised in August 2018, which builds a layered proximity graph and whose abstract claims logarithmic complexity scaling. Index limits are real and low: pgvector permits up to 16,000 dimensions in a vector column, but its HNSW and IVFFlat indexes cover only up to 2,000 of them for that type.
Scoring. Neighbours come back ranked by a distance metric. pgvector exposes six — L2, inner product, cosine, L1, Hamming and Jaccard — and the correct one is whichever the embedding model was trained under. A mismatch here does not error; it silently returns a different neighbourhood.
Why it decides whether you get read
An agent answering a question does not read your page. It reads a handful of spans that an index selected, then writes from them. Two consequences follow directly.
First, the passage is the unit of competition. A page can be authoritative, correctly structured and well linked, and still lose because the specific span containing the answer also contained three other subjects, so its vector sits between four meanings instead of on one.
Second, similarity is not correctness. The index returns what is nearest, not what is right. A passage that discusses your pricing model in general terms is geometrically close to a question about your pricing model in particular, and will be returned for it — after which the model writes an answer that sounds sourced and is wrong. That failure is invisible from the retrieval side; it looks like a successful hit.
Failure modes worth naming
- Multi-topic chunks. The commonest and the most fixable. One subject per section, stated in the heading.
- Orphaned qualifiers. A caveat two paragraphs above the claim is in a different chunk and does not travel with it. Write the qualifier into the same sentence as the claim.
- Exact identifiers. Vector similarity is strong on paraphrase and weak on literal strings. Product codes, version numbers, error codes and prices need to appear as plain text a lexical matcher can hit, which is why production systems combine vector and keyword retrieval rather than choosing.
- Dimension mismatch. Content embedded with one model and queried with another produces coordinates in unrelated spaces. The system returns results; they are noise.
- Stale index. An edit to your page changes nothing until the span is re-embedded and re-indexed. Correcting a wrong fact is not the same event as the correction reaching answers.
- Over-pruned recall. Approximate means approximate. Aggressive index settings drop true nearest neighbours, and the drop is silent.
What to do about it
Write sections that survive being lifted out. Give each one subject and a heading that states the claim rather than labelling the topic. Keep identifiers literal. Repeat the qualifier next to the number it qualifies. Then check what is actually being said about you in answers — a wrong claim attributed to your page is usually traceable to one retrievable span that reads badly alone, and editing that span is the fix.
Frequently asked questions
Is vector search the same as semantic search?
Vector search is the dominant implementation of semantic search, not a synonym for it.
Does it replace keyword search?
No. It is weak exactly where keyword search is strong, on exact identifiers, which is why hybrid retrieval is standard.
Which distance metric should be used?
The one the embedding model was trained under. pgvector documents six; picking the wrong one degrades results without raising an error.
What is the practical dimension ceiling?
Set by the index, not the model: pgvector allows 16,000 dimensions in a vector column but its HNSW and IVFFlat indexes cover only 2,000 of them for that type (4,000 for halfvec).
Terms related to Vector search
Vectors of floating point numbers whose distance measures relatedness, the representation underneath semantic search and retrieval.
Retrieval by meaning rather than by matching strings, what it is genuinely better at, and the class of query where it reliably fails.
The architecture that retrieves documents at query time and has a model write from them, and the reason your page can be quoted without ever being trained on.
The token budget a model can reference in one request, what counts against it, and why a bigger window does not remove the need to retrieve selectively.