MMLU-Pro (knowledge) | The Metacognition Bench

Operating-point board

MMLU-Pro (knowledge) · Prospective · F_β β

#	Model	F_β ?	Prec ?	Rec ?	Task acc ?
1	GPT-5.2	0.861	0.82	0.91	0.73
2	Gemini 2.5 Pro	0.822	0.73	0.94	0.67
3	Claude Sonnet 4.5	0.819	0.78	0.87	0.70
4	Gemini 3 Pro	0.814	0.75	0.89	0.66
5	DeepSeek R1	0.799	0.72	0.90	0.65
6	Gemini 3 Flash	0.778	0.65	0.98	0.61
7	Claude Haiku 4.5	0.744	0.73	0.76	0.65
8	Claude 3.5 Sonnet	0.705	0.58	0.89	0.54
9	DeepSeek Chat	0.693	0.55	0.95	0.51
10	GPT-4o	0.659	0.51	0.94	0.47
11	Mistral Medium 3.1	0.648	0.52	0.87	0.48
12	Gemini 2.5 Flash	0.636	0.48	0.96	0.46
13	Gemini 2.0 Flash	0.623	0.45	1.00	0.45
14	GPT-4o mini	0.595	0.43	0.95	0.40
15	Mistral Small 3.2	0.573	0.41	0.95	0.39
16	Qwen 2.5 72B	0.570	0.46	0.76	0.41
17	Llama 3.1 70B	0.549	0.39	0.95	0.36
18	Llama 3.3	0.525	0.36	0.94	0.34
19	Claude 3 Haiku	0.512	0.35	0.93	0.35
20	Qwen 2.5 Coder 32B	0.405	0.25	1.00	0.25

Observed correlation

0.041

mean τ-b · 0 = no signal

Baseline correlation

0.001

permutation null · ≈ 0

Model pairs

190

unit = pair, not model

Significant pairs

35.8%

p<0.05 after FDR · not effect size

Observed model pairs Base-rate-matched null Calibration-preserving null 5%-95% observed: -0.030 to 0.130