Constrained Decoding
Constrained decoding forces a language model’s output to satisfy a formal rule, such as a regular expression, a grammar, or a JSON Schema, by changing which tokens the sampler may pick at each step.
-
-
LLM Decoding and Structured Output
A language model outputs a distribution, and the decoder picks the tokens. Learn temperature and top-p, token masking for structured output, and how to evaluate it.