Do human reviewers score every response during RLHF training?
Not necessarily. A learned reward model can score additional generated responses after being trained on human comparisons, reducing the need for direct human review each time.
Explore This Question
Search for additional explanations using these tools:
More FAQs in What is a reward model, and how is it trained using human preferences?
They serve different roles: the assistant generates responses, while the reward model estimates a feedback score. Some approaches share foundations, but the functions remain distinct.
It is an imperfect proxy for the intended goal. An optimized model may find response patterns that earn high scores without delivering the desired quality.
Still have questions?
Browse all our FAQs or reach out to our support team