What is supervised fine-tuning?

What is supervised fine-tuning?

Supervised fine-tuning, or SFT, trains an existing model on examples of desired behavior. Each example pairs an input with an appropriate target output. The model's parameters are adjusted so that similar inputs are more likely to produce responses matching those demonstrations.

Learning from completed examples

A customer-support dataset might pair “My order arrived damaged” with a response that acknowledges the issue, asks for required information, and follows the company's return process.

Training rewards the target response pattern. Good examples therefore need correct content, suitable tone, and consistent handling of missing information. Contradictory demonstrations can teach inconsistent behavior.

OpenAI's SFT guide describes this general approach as training with example inputs and known good outputs.

How SFT differs from prompting

Prompting supplies instructions or examples during a request without normally changing learned weights. SFT changes model parameters through a training procedure.

Retrieval is different again: it brings relevant documents into the request context. For frequently changing facts, retrieval may be a better fit than repeatedly embedding information through training.

Decide whether the improvement is real

Keep representative examples outside the training set and compare the fine-tuned model with the starting model. Test variations in wording and unfamiliar cases, not just near-copies of demonstrations.

SFT can improve consistency and task behavior, but it does not guarantee truthfulness or eliminate the need for clear instructions. Its value depends on the quality of the examples and measurable improvements in the target workflow.