Instruction Tuning

Instruction tuning is additional model training using examples organized as instructions and desired responses. It aims to make the model better at following task requests. It changes learned parameters, or a trainable subset such as adapters, rather than merely supplying a new request at inference time.

Teach a response pattern across examples

Imagine training examples that ask a model to summarize a policy, preserve exceptions, and identify missing information. Each example pairs a task with an appropriate response. Learning across such examples can improve how the model responds to related instructions. The FLAN research studied instruction-based fine-tuning and generalization to tasks outside its fine-tuning mixture.

The examples must embody the behavior wanted. Responses that confidently invent exceptions can teach an undesirable pattern. Training loss alone does not establish correct behavior on new policies, languages, or unusual requests.

Similar inputs can play different roles

Pasting an instruction and examples into one prompt usually leaves parameters unchanged. Instruction tuning performs training updates. Retrieval supplies selected external material for use in a request; it is not automatically instruction tuning.

Instruction tuning is one kind of fine-tuning, and not all fine-tuning uses instruction-response examples. Preference-based training is another related approach with a different learning signal. None guarantees that every future request will be followed correctly. Evaluate held-out tasks and retain application-level permission and correctness checks.

Reference: Wei et al.: Finetuned Language Models Are Zero-Shot Learners.