Query Rewriting

Query rewriting transforms the text a user typed into one or more search strings before retrieval. The user’s words are often a poor search query: a follow-up such as “what about the pro plan?” depends on earlier turns, a short query shares no vocabulary with the answering document, and a compound question needs two different documents. A rewrite step, usually a language model call, tries to produce queries the retriever can use.

Common transformations

Transformation	What it addresses	Typical failure
Clarification / standalone rewrite	A follow-up that depends on conversation history is rewritten into a self-contained question	Resolves a pronoun to the wrong entity, or adds a fact the user never stated
Normalization	Spelling, casing, separators, units	Rewrites an identifier or removes a qualifier
Expansion	Adds synonyms or related terms, or several paraphrases, to bridge vocabulary gaps	Adds terms that pull in a different topic, and each extra query costs retrieval and fusion work
Decomposition	Splits a multi-part question into sub-queries retrieved separately	Loses the relationship between the parts, or splits something that needed to stay joined
HyDE	A model writes a hypothetical answer document and its embedding is used as the query	The invented document can contain false specifics that steer retrieval toward what it invented

Anantha and colleagues released QReCC for rewriting conversational questions into standalone form. Gao and colleagues propose HyDE: the generated document is “unreal” and may contain false details, and the method relies on the encoder to map it near genuine documents that are relevant. HyDE is suited to dense retrieval, where an answer-shaped text lies closer to the answering passage than the short question does.

A traced example

The program below is a hypothetical word-overlap retriever over four short documents:

import re
DOCS = {
 "d1": "error E4029 on plan pro: reset the gateway",
 "d2": "billing plan limits for pro accounts",
 "d3": "gateway timeout troubleshooting guide",
 "d4": "how to reset a password",
}
def tokens(text): return set(re.findall(r"[a-z0-9]+",text.lower()))
def rank(query):
    q = tokens(query)
    scores = {name:len(q & tokens(text)) for name,text in DOCS.items()}
    return sorted(scores.items(),key=lambda pair:(-pair[1],pair[0]))
for q in ("e4029 pro reset","e4029 pro reset gateway","how do I fix a gateway problem"):
    print(q,rank(q))

e4029 pro reset [('d1', 3), ('d2', 1), ('d4', 1), ('d3', 0)]
e4029 pro reset gateway [('d1', 4), ('d2', 1), ('d3', 1), ('d4', 1)]
how do I fix a gateway problem [('d4', 2), ('d1', 1), ('d3', 1), ('d2', 0)]

The query “e4029 pro reset” returns d1 first. A careful expansion, “e4029 pro reset gateway”, also returns d1 first, with d2, d3, and d4 tied on one matching word; the code breaks ties by document id, so d2 is second. A rewrite that paraphrases the identifier away, “how do I fix a gateway problem”, returns d4 first and d1 second. The paraphrase reads better as English and is a worse query: it lost the exact token that identified the document. In a real system a language model rewriting “ERR-4029” into “that error” does exactly this.

Why failure modes matter more than gains

A rewrite is an unobserved change to the question that everything downstream trusts. If it drops the identifier, adds an unstated constraint or resolves a reference wrongly, the retrieved passages, the reranked list and the generated answer all look coherent while answering a different question. Three practices follow:

• Keep the original. Retrieve with the original query as one leg and add rewritten queries alongside it, so a bad rewrite cannot remove what the original would have found.
• Protect exact terms. Identifiers, quoted strings, numbers and negations should survive a rewrite verbatim. A check that every such token in the original appears in the rewrite catches the most damaging errors cheaply.
• Measure by query type. A rewriter helps follow-ups and vocabulary mismatches and can hurt identifier lookups. Compare retrieval quality with and without it on each slice of a benchmark, and log the original and rewritten text per query.

Related terms

Query rewriting changes the query before retrieval. Hybrid search runs several retrievers on a query and fuses results, and reranking reorders candidates afterward. Rewriting can be combined with either, since a multi-query expansion produces several lists that are fused like lists from different retrievers.

References

• Gao et al., Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE) — https://arxiv.org/abs/2212.10496