LegalBench (legal reasoning) | The Metacognition Bench

Operating-point board

LegalBench (legal reasoning) · Prospective · F_β β

#	Model	F_β ?	Prec ?	Rec ?	Task acc ?
1	Gemini 3 Pro	0.927	0.88	0.98	0.87
2	Mistral Medium 3.1	0.863	0.85	0.88	0.86
3	Llama 3.1 70B	0.861	0.88	0.84	0.88
4	Gemini 3 Flash	0.859	0.89	0.83	0.88
5	Mistral Small 3.2	0.854	0.85	0.86	0.85
6	GPT-5.2	0.852	0.85	0.85	0.85
7	DeepSeek R1	0.851	0.88	0.83	0.87
8	Gemini 2.5 Pro	0.822	0.87	0.78	0.87
9	Claude 3.5 Sonnet	0.821	0.89	0.76	0.86
10	Gemini 2.5 Flash	0.810	0.84	0.78	0.84
11	Claude Sonnet 4.5	0.795	0.87	0.73	0.86
12	Llama 3.3	0.763	0.88	0.68	0.88
13	Claude Haiku 4.5	0.742	0.87	0.65	0.86
14	Gemini 2.0 Flash	0.734	0.81	0.67	0.85
15	DeepSeek Chat	0.674	0.83	0.57	0.84
16	Claude 3 Haiku	0.660	0.89	0.53	0.91
17	Qwen 2.5 72B	0.626	0.83	0.50	0.84
18	GPT-4o mini	0.600	0.86	0.46	0.85
19	GPT-4o	0.576	0.82	0.44	0.83

Observed correlation

0.024

mean τ-b · 0 = no signal

Baseline correlation

-0.000

permutation null · ≈ 0

Model pairs

171

unit = pair, not model

Significant pairs

25.7%

p<0.05 after FDR · not effect size

Observed model pairs Base-rate-matched null Calibration-preserving null 5%-95% observed: -0.060 to 0.127