Compare models | The Metacognition Bench

Side by side, per benchmark

selected models (provider colors) all other model/condition points equal relative confidence and pass rate

SQuAD (factual recall) · Prospective · F_β β

#	Model	F_β ?	Prec ?	Rec ?	Task acc ?
1	Gemini 3 Pro	0.785	0.77	0.80	0.65
2	Gemini 3 Flash	0.782	0.70	0.88	0.59
3	Gemini 2.5 Pro	0.764	0.71	0.83	0.58
4	Claude Sonnet 4.5	0.753	0.78	0.72	0.59
5	GPT-4o	0.739	0.69	0.80	0.54
6	DeepSeek R1	0.733	0.76	0.70	0.57
7	Claude Haiku 4.5	0.716	0.79	0.65	0.48
8	Mistral Medium 3.1	0.703	0.65	0.77	0.52
9	GPT-4o mini	0.686	0.56	0.88	0.45
10	Gemini 2.0 Flash	0.683	0.58	0.82	0.45
11	GPT-5.2	0.673	0.84	0.56	0.62
12	Gemini 2.5 Flash	0.670	0.58	0.79	0.45
13	Mistral Small 3.2	0.666	0.57	0.79	0.44
14	Llama 3.1 70B	0.643	0.65	0.64	0.45
15	Claude 3.5 Sonnet	0.643	0.79	0.54	0.56
16	Llama 3.3	0.593	0.64	0.55	0.45
17	Qwen 2.5 72B	0.572	0.74	0.47	0.47
18	DeepSeek Chat	0.567	0.78	0.45	0.55
19	Claude 3 Haiku	0.554	0.55	0.56	0.40
20	Qwen 2.5 Coder 32B	0.476	0.31	1.00	0.31

Match accuracy controls for the performance base-rate gap

selected pairs

1/ 171

selected mean tau

-0.014

All-pairs mean

+0.037

selected p<0.05

0(0%)

all model pairs (observed) base-rate-matched null calibration-preserving null selected pair (filled = p<0.05) selected mean all-pairs mean