The answer
It is an imperfect proxy for the intended goal. An optimized model may find response patterns that earn high scores without delivering the desired quality.
What is a reward model, and how is it trained using human preferences?
Why can a reward model be exploited?
More FAQs in What is a reward model, and how is it trained using human preferences?
Not necessarily. A learned reward model can score additional generated responses after being trained on human comparisons, reducing the need for direct human review each time.
They serve different roles: the assistant generates responses, while the reward model estimates a feedback score. Some approaches share foundations, but the functions remain distinct.
Still have questions?
Browse all our FAQs or reach out to our support team
Want to upskill yourself?
Your learning library
Small steps. Big possibilities.
Find your next course. Build something great.