Why it matters

Arena-Hard correlates with human without cost. Understanding shapes eval choice.

Advertisement

The architecture

500 curated prompts.

GPT-4 judges pairwise.

Bradley-Terry rating.

Arena-Hard flow500 promptschallengingGPT-4 judgepairwise scoringBT ratingleaderboardgithub.com/lmarena/arena-hard-auto; correlates ~90% with human Arena
Arena-Hard.
Advertisement

How it works end to end

GPT-4o / Sonnet judges.

Bradley-Terry model.

Auto-refreshed prompts.

Style Control variant.