Language Model Objectives and Limits
A language model learns the conditional distribution of the next token. That one objective explains both its range and why its answers are not reliably correct.
A language model learns the conditional distribution of the next token. That one objective explains both its range and why its answers are not reliably correct.