Meaning-based retrieval: embeddings, hybrid ranking and evaluation

Keyword search returns documents that contain the words you typed. Semantic search returns documents that mean what you asked, even when they use different words. For agents and for retrieval-augmented generation it has become the default retrieval layer, which makes it easy to forget that it fails in ways keyword search does not. This page explains how embeddings work, where meaning-based retrieval breaks, why most production systems pair it with keywords, and how to measure whether any of it is working.

The gap keyword search leaves

Lexical search matches terms after normalizing them, then ranks documents by how often and how distinctively those terms appear. It is fast, predictable and very good at exact things. It is also literal. A help article titled "Terminating your subscription" shares no meaningful word with the question "how do I cancel my plan?", so a pure keyword engine may not rank it at all. Users and agents do not phrase questions the way documents are written, and that vocabulary mismatch is the problem semantic search was built to address.

Keyword search still wins on its home ground: product codes, error identifiers such as E-4012, people's names, clause numbers, quoted phrases and rare technical terms. Any design that drops it entirely gives up those strengths.

How embeddings capture meaning

An embedding model converts a piece of text into a vector, a fixed-length list of numbers. The model is trained so that texts with similar meaning end up close together in that space and unrelated texts end up far apart. Similarity then becomes a geometric question, usually answered with cosine similarity or a dot product.

embed("Terminating your subscription")  -> [0.12, -0.40, 0.88, ...]
embed("how do I cancel my plan?")       -> [0.10, -0.37, 0.91, ...]
embed("Quarterly revenue by region")    -> [-0.66, 0.21, 0.05, ...]

cosine(first, second) is high    close in meaning, no shared words
cosine(first, third)  is low     different topic

Three practical consequences follow. Queries and documents must be embedded with the same model, because vectors from different models are not comparable. Changing the model means re-embedding the whole collection. And a model only knows the language it was trained on, so specialist vocabulary from law, medicine or your own product may be represented poorly.

Documents are rarely embedded whole. They are split into passages, or chunks, small enough that one vector can stand for one idea. Chunk size is a genuine trade-off: chunks that are too small lose context, while chunks that are too large blur several topics into one vector. Keeping the document title and section heading with each chunk, plus metadata such as source, date and project, makes both retrieval and citation far more reliable.

The retrieval pipeline

1. Index time. Split documents into chunks, embed each chunk, and store the vectors with their metadata in a vector index.
2. Query time. Embed the question with the same model and ask the index for the nearest chunks.
3. Filter. Restrict results by metadata such as project, language, date range and, above all, access rights. Apply permission filters before anything reaches the model; never rely on the model to ignore text it should not have seen.
4. Rerank. Optionally pass the top candidates to a reranker, which reads the query and each passage together and orders them more precisely than vector distance alone.
5. Return with sources. Hand back passages with a pointer to where each one came from, so the answer can cite them and a reader can check.

Where semantic search goes wrong

• Exact identifiers. Two invoice numbers that differ by one digit look almost identical to an embedding model.
• Negation and constraints. "Contracts without automatic renewal" tends to retrieve contracts about automatic renewal, because the topic dominates the vector.
• Related but not responsive. A passage on the same subject that does not answer the question can outscore a terse passage that does.
• Always something. Nearest-neighbor search always returns its top results, even when none of them is relevant. Without a threshold or a check, the system cannot say "nothing found", and an agent will happily build an answer on weak matches.
• Uncalibrated scores. Similarity scores are not probabilities and drift between models and collections, so a fixed cut-off copied from somewhere else will not transfer.
• Stale vectors. Documents edited after embedding, or a model upgrade applied to only part of the collection, leave the index quietly inconsistent.

Hybrid search: use both signals

Hybrid search runs a keyword query and a vector query side by side and merges the two ranked lists. A simple and robust way to merge is reciprocal rank fusion. It uses only positions in each list, so you never have to make keyword scores and vector scores comparable:

keyword results:  D7, D2, D9, D4
vector results:   D2, D5, D7, D1

fused score(d) = sum over lists of 1 / (k + rank of d in that list)
                 (k is a smoothing constant)

fused order:      D2, D7, D5, D9, ...

D2 and D7 rise because both methods found them; D5 survives on meaning alone and D9 on keywords alone. Weighted blending of raw scores is the alternative, but it needs careful normalization. Many systems also route queries: a string that looks like an identifier leans on keywords, a long natural-language question leans on vectors. A reranker applied to the merged list often adds more than further tuning of either retriever.

How to evaluate retrieval

Without measurement, semantic search is a matter of opinion. Build a small test set from real questions, ideally taken from logs or from the people who will use the system, and label which passages actually answer each one. Include the hard cases on purpose: paraphrases with no shared words, identifier lookups, negations, and questions the collection cannot answer.

query:     "which supplier contracts renew automatically?"
relevant:  contracts/supplier-a.pdf#page=7
           contracts/supplier-d.pdf#page=3
case:      paraphrase (clause says "shall extend for successive terms")
expect:    both passages in the top 5; no unrelated contract above them

query:     "contracts signed in the Ankara office"
relevant:  none
case:      unanswerable in this collection
expect:    empty result or an explicit "nothing found"

Standard measures help you compare runs: recall at k (did the right passage appear in the top k results), precision, mean reciprocal rank, and nDCG for graded relevance. Run keyword, vector and hybrid configurations against the same set and look at failures by case type, not just the average. Evaluate retrieval and the final answer separately, since a good passage can still be misquoted. Rerun the set whenever you change chunking, the embedding model, filters or the reranker. The guide to evaluating search and the article on evaluating web search on your own queries go deeper on method, and judging recall and accuracy discusses what such numbers can and cannot tell you.

Semantic retrieval inside GoCux

In GoCux, knowledge search answers a question scoped to a single project's documents. After a document is imported, full-text search works on it straight away; semantic retrieval adds its results when it is available, so keywords and meaning work together rather than one replacing the other. Results come back as passages with citations. A citation points to the evidence; it does not prove the claim, and the reader still checks that the passage says what the answer says it does. An empty result counts as a valid answer, and an unavailable search is reported as unavailable, never dressed up as "nothing found". More on knowledge search and on citations that hold up.

Checklist for a semantic search feature

• Do you keep a keyword path for identifiers, names and exact phrases?
• Is the same embedding model used for documents and queries, and is the model version recorded with each vector?
• Does each chunk carry its document title, section and source?
• Are permission filters applied before results reach the model?
• Can the system return "nothing found" instead of its least bad matches?
• Do you have a labelled test set with paraphrases, negations and unanswerable questions?
• Do you rerun that set after every change to chunking, model, filters or ranking?