Why it matters

GRPO enables reasoning training. Understanding shapes modern alignment.

Advertisement

The architecture

Sample N completions per prompt.

Score each.

Advantage: (r - mean) / std within group.

GRPO FT flowN samplesper promptReward eachverifier or RMGroup advantagenormalize + PPODeepSeek R1 uses this for reasoning; verifier-based rewards + no value model
GRPO FT.
Advertisement

How it works end to end

No value model = simpler.

Group N = 8-16.

Verifier for reasoning tasks.