Evaluating Retrieval and Answers Separately
One accuracy number cannot say whether a RAG failure came from retrieval, context selection, or generation. Score each stage, audit the judges, and diagnose by stage.
-
-
Hybrid Retrieval and Reranking
Sparse and dense retrievers fail on different queries. Learn how to fuse them, where a reranker helps or hurts, how to rewrite queries without losing exact terms, and how to benchmark each stage.
-
Relevance Judgments and Pooling
Retrieval metrics depend on relevance judgments. Pooling makes judging feasible, but a new system can retrieve relevant documents that earlier systems never put in the pool.
-
Embeddings and Retrieval Representations
Retrieval quality is set by the representation: BM25 matches terms, dense embeddings match what training taught them. Learn the objectives, the failure modes, and how to measure them.