SQuAD (factual recall) | The Metacognition Bench

Operating-point board

SQuAD (factual recall) · Prospective · F_β β

#	Model	F_β ?	Prec ?	Rec ?	Task acc ?
1	Gemini 3 Pro	0.785	0.77	0.80	0.65
2	Gemini 3 Flash	0.782	0.70	0.88	0.59
3	Gemini 2.5 Pro	0.764	0.71	0.83	0.58
4	Claude Sonnet 4.5	0.753	0.78	0.72	0.59
5	GPT-4o	0.739	0.69	0.80	0.54
6	DeepSeek R1	0.733	0.76	0.70	0.57
7	Claude Haiku 4.5	0.716	0.79	0.65	0.48
8	Mistral Medium 3.1	0.703	0.65	0.77	0.52
9	GPT-4o mini	0.686	0.56	0.88	0.45
10	Gemini 2.0 Flash	0.683	0.58	0.82	0.45
11	GPT-5.2	0.673	0.84	0.56	0.62
12	Gemini 2.5 Flash	0.670	0.58	0.79	0.45
13	Mistral Small 3.2	0.666	0.57	0.79	0.44
14	Llama 3.1 70B	0.643	0.65	0.64	0.45
15	Claude 3.5 Sonnet	0.643	0.79	0.54	0.56
16	Llama 3.3	0.593	0.64	0.55	0.45
17	Qwen 2.5 72B	0.572	0.74	0.47	0.47
18	DeepSeek Chat	0.567	0.78	0.45	0.55
19	Claude 3 Haiku	0.554	0.55	0.56	0.40
20	Qwen 2.5 Coder 32B	0.476	0.31	1.00	0.31

Observed correlation

0.037

mean τ-b · 0 = no signal

Baseline correlation

0.000

permutation null · ≈ 0

Model pairs

171

unit = pair, not model

Significant pairs

46.2%

p<0.05 after FDR · not effect size

Observed model pairs Base-rate-matched null Calibration-preserving null 5%-95% observed: -0.026 to 0.127