OmniMath (advanced math) | The Metacognition Bench

Operating-point board

OmniMath (advanced math) · Prospective · F_β β

#	Model	F_β ?	Prec ?	Rec ?	Task acc ?
1	Gemini 3 Pro	0.941	0.89	0.99	0.88
2	GPT-5.2	0.923	0.88	0.97	0.85
3	DeepSeek R1	0.840	0.72	1.00	0.70
4	Gemini 2.5 Pro	0.830	0.71	1.00	0.70
5	Gemini 3 Flash	0.830	0.71	1.00	0.71
6	Gemini 2.5 Flash	0.786	0.65	1.00	0.64
7	Claude Haiku 4.5	0.747	0.67	0.85	0.58
8	Claude Sonnet 4.5	0.718	0.63	0.83	0.57
9	DeepSeek Chat	0.713	0.59	0.89	0.56
10	Gemini 2.0 Flash	0.642	0.47	0.99	0.47
11	Mistral Small 3.2	0.631	0.49	0.90	0.44
12	Mistral Medium 3.1	0.601	0.83	0.47	0.55
13	Qwen 2.5 72B	0.568	0.50	0.65	0.34
14	Llama 3.3	0.537	0.44	0.68	0.33
15	GPT-4o	0.524	0.40	0.76	0.30
16	GPT-4o mini	0.469	0.31	0.99	0.30
17	Llama 3.1 70B	0.461	0.36	0.65	0.21
18	Claude 3 Haiku	0.291	0.20	0.51	0.13

Observed correlation

0.025

mean τ-b · 0 = no signal

Baseline correlation

-0.000

permutation null · ≈ 0

Model pairs

153

unit = pair, not model

Significant pairs

20.3%

p<0.05 after FDR · not effect size

Observed model pairs Base-rate-matched null Calibration-preserving null 5%-95% observed: -0.082 to 0.175