Reinforcement Learning with Verifiable Rewards

Reinforcement learning with verifiable rewards (RLVR) is a post-training method in which a model is reinforced only when an external program can confirm — deterministically and without human judgement — that its answer is correct. Instead of training a reward model to guess what humans would prefer, RLVR scores outputs with an exact-answer check, a unit test, a schema validator or a proof checker.^[1][4]

Background

Reinforcement learning from human feedback (RLHF) made chatbots alignable but carries a familiar cost: preference data must be collected and annotated, and the resulting reward model is only an approximation of human taste. It is subjective, expensive to refresh, and hard to audit after the fact.^[5]

RLVR grew out of standard reinforcement learning with deterministic reward functions — the technique itself is old; what changed is applying it at scale to language-model post-training. DeepSeekMath (2024) introduced group relative policy optimisation (GRPO) for mathematical reasoning, and DeepSeek-R1 (January 2025) paired rule-based rewards — correct answers, valid code, well-formed structured output — with GRPO to train a frontier-class reasoning model using almost no human preference data.^[1][2] That result made verifiable rewards the default recipe for reasoning models across the industry, and by 2026 most commercial "thinking" modes trace their post-training to some variant of the loop.^[3]

Key Concepts and Technology

The core object is the verifier: code that maps a model output to a reward, typically 0 or 1. Because the same input always produces the same reward, training feedback is deterministic, reproducible and auditable — properties that learned reward models cannot guarantee.^[4]

RLVR works cleanly where ground truth exists: mathematics and logic, code checked against unit tests, formal proofs, structured outputs validated against JSON schemas, and agent trajectories checked against environment state. Tasks dominated by taste, empathy or open-ended judgement still need human feedback, rubric scoring or model-based judges.^[4]

Optimisation usually runs through GRPO, which samples a group of completions for each prompt and reinforces outputs relative to their group-mates, or through PPO with a scalar verifier reward. Two caveats temper the results. First, research suggests much of the measured gain is search compression — the model already could solve the problem and now reaches it in one attempt — rather than a wholly new capability.^[3] Second, verifiers can be gamed: models learn shortcuts that satisfy the checker (reward hacking), and some training setups show sensitivity to spurious or even random reward signals, so gains must be validated on held-out, distribution-shifted test sets.^[3][6]

Applications and Impact

RLVR is now the backbone of reasoning models used for maths tutoring, scientific question answering, code generation and test-driven development, where correctness is checkable by construction. Enterprises apply it to document extraction and structured-data tasks with schema validators, to data-pipeline jobs with row-level assertions, and increasingly to agents, where a reward can encode whether required evidence was gathered, permitted tools were used and the workflow completed correctly.^[4] Its economic appeal is direct: verification replaces annotation, and verifier logic can be revised without retraining a reward model.

>See Also

• GRPO
• RLHF
• Reward Modeling
• Reward Hacking
• Reasoning Models
• Reinforcement Learning
• DeepSeek