LLM-as-a-Judge
A model can grade thousands of answers overnight. It can also favor longer answers or the one shown first. Check the judge before trusting its scores.
-
-
ReAct Pattern
Think, act, observe, repeat. The loop lets an agent adjust to what its tools return, and it keeps going until something tells it to stop.
-
Groundedness
An answer can cite real passages and still claim something they do not say. Groundedness asks, claim by claim, whether the evidence supports the answer.
-
Chunking
A passage retrieved without its heading can say the opposite of what the document means. How documents are split decides what search can find.
-
Reranking
Fast search finds a hundred plausible passages; a slower model decides which five the answer should rest on. Reranking cannot rescue what was never found.
-
Structured Output
A supported schema constrains a successful response. Completion checks and semantic validation are still needed before the application accepts its values.
-
OpenAI API (ML API)
Use the OpenAI API through authenticated requests, handle typed outputs, and separate answer validation, token billing, latency, and rate-limit controls.
-
LLMs (Large Language Models)
Many generative LLMs predict text one token at a time. Their training enables useful language tasks, while generated answers still need checks against the task and available evidence.
-
Model Cascade
A model cascade answers each request with a cheap model first and escalates to a more capable, more expensive model only when a check says the cheap answer is not good enough.
-
Cost per Successful Task
Cost per successful task is the total spend on a workload divided by the number of tasks that finished with a correct outcome.
-
Semantic Cache
A semantic cache stores earlier requests with their answers and, for a new request, returns the stored answer of the nearest earlier request if the two are similar enough.
-
Orchestrator-Worker Pattern
In the orchestrator-worker pattern, one lead agent (the orchestrator) breaks a task into subtasks, hands each to a worker agent, and combines what the workers return.
-
Effect Ledger
An effect ledger is an append-only record of the state changes that an agent’s tool calls actually caused: one entry per refund issued, record updated, email sent, or escalation opened.
-
Knowledge Conflict
A knowledge conflict occurs when two sources of information that a language model application uses disagree.
-
Citation Precision and Recall
An answer that cites sources makes two kinds of claims: that each statement is supported, and that the cited passage is where the support lies.
-
Hybrid Search
Hybrid search runs a lexical retriever and a vector retriever on the same query and combines their results into one ranked list.
-
Cross-Encoder
A cross-encoder is a model that reads a query and a candidate document together, as one input, and outputs a single relevance score.
-
Query Rewriting
Query rewriting transforms the text a user typed into one or more search strings before retrieval.
-
Prompt Fingerprint
A prompt fingerprint is a short hash computed from everything that defines how a model call is configured, so that two calls can be compared for “same prompt release or not” without reading the text.
-
Endpointing
Endpointing is the decision that a speaker has finished a turn, so the system can stop listening, finalize the transcript, and respond.
-
Word Error Rate
Word error rate (WER) is the standard score for speech recognition.
-
Fréchet Inception Distance (FID)
FID scores how close a set of generated images is to a set of real images.
-
ControlNet
ControlNet adds spatial control, such as edges, depth maps or human poses, to a pretrained text-to-image diffusion model without retraining that model.
-
Classifier-Free Guidance
Classifier-free guidance (CFG) is a sampling-time technique for conditional diffusion models.
-
Temporal Consistency
Temporal consistency is how well a generated video holds together from one frame to the next.
-
Object Hallucination
Object hallucination is when a vision-language model (a language model that reads an image and answers in text) mentions or confirms an object that is not in the image.
-
Neural Audio Codec
A neural audio codec is a learned compressor for audio.
-
Modality Gap
The modality gap is the observation that, in a trained contrastive model such as CLIP, image embeddings and text embeddings occupy two separate regions of the shared space, even though matched image-caption pairs are close to each other.
-
Visual Connector
A visual connector is the trainable module that turns the output of a vision encoder into vectors a language model can read as part of its input sequence.
-
Data Decontamination
Data decontamination is the removal, from a training set, of examples that also appear in an evaluation set, including examples that are copies with small changes.
-
Late Interaction (ColBERT and MaxSim)
Retrieval models sit on a spectrum of how early the query and the document are allowed to look at each other.
-
BM25
BM25 is a lexical scoring function: it ranks documents by how well the words of a query match the words of each document, with no learned vectors involved.
-
Reciprocal Rank Fusion
Reciprocal rank fusion (RRF) combines several ranked lists of documents into one.
-
Needle-in-a-Haystack Test
The needle-in-a-haystack test checks whether a language model can use a fact placed somewhere inside a long input.
-
Attention Sink
In many trained decoder-only transformers, a large share of the attention in many heads goes to the first few tokens of the sequence, whatever those tokens say.
-
Greedy Decoding and Beam Search
A language model gives a probability distribution over the next token.
-
Constrained Decoding
Constrained decoding forces a language model’s output to satisfy a formal rule, such as a regular expression, a grammar, or a JSON Schema, by changing which tokens the sampler may pick at each step.
-
Top-k and Top-p Sampling
Top-k and top-p (nucleus) sampling are two ways of cutting off the unlikely tail of a language model’s next-token distribution before choosing a token at random.
-
Pass@k and Pass^k
Both metrics start from the same experiment: run a task several times with a stochastic system (a language model sampled at nonzero temperature, or an agent that makes different tool calls on different runs) and record success or failure for each run.
-
In-Context Learning
In-context learning (ICL) is a language model changing its behavior on a task because of the text in its prompt, with no change to its weights.
-
Exposure Bias
Exposure bias is the mismatch between how an autoregressive language model is trained and how it is used.
-
Chain-of-Thought Prompting
Chain-of-thought (CoT) prompting makes a language model write intermediate steps before its final answer, instead of answering directly.
-
Text-to-Speech (TTS)
Understand speech synthesis from text, including pronunciation, timing, intermediate representations, and listening-based checks.
-
Speech Recognition (ASR)
Understand automatic speech recognition, how it differs from translation and speaker attribution, and how to check transcripts.
-
Diffusion
Understand diffusion generation through noisy representations, learned denoising, conditioning, and the limits of image plausibility.
-
Multimodal
Understand multimodal inputs and outputs, and distinguish a model capability from an application assembled from several models.
-
Hallucination
Understand hallucination and confabulation as unsupported or false generation, and evaluate claims against sources and execution evidence.
-
AI Agent
Understand model-directed agents, their interaction loop, and why permissions, stopping rules, and execution evidence still matter.
-
Tool Call
Understand a tool call as a structured operation request and separate argument validation, authorization, execution, and confirmation.
-
Instruction Tuning
Understand instruction tuning as training on task instructions and responses, and distinguish it from prompting and retrieval.