GLOSSARY
RLHF (Reinforcement Learning from Human Feedback)
The training step that makes raw language models polite, helpful and safe — by teaching them from thousands of human judgments about which answers are better.
A base model predicts the next token; it has no notion of being helpful. RLHF closes that gap: humans rank pairs of model answers, a reward model learns those preferences, and reinforcement learning nudges the model toward answers people actually prefer — polite, on-topic, honest about uncertainty and refusing harmful requests.
It is the single biggest reason ChatGPT feels different from a raw LLM. The tradeoffs are real: RLHF can make models cautious to the point of evasiveness, bias answers toward pleasant agreement (sycophancy), and wash out some raw capability. RLAIF and DPO are newer variants that reduce the human-labor cost.