Positioning spread
| Benchmark | Task acc | Confidence | F₁ | Leans |
|---|---|---|---|---|
| SQuAD (factual recall) | 0.45 | 0.45 | 0.64 | -0.01 calibrated |
| MMLU-Pro (knowledge) | 0.36 | 0.90 | 0.55 | +0.54 overconfident |
| LegalBench (legal reasoning) | 0.88 | 0.84 | 0.86 | -0.04 calibrated |
| MathBench (competition math) | 0.63 | 0.96 | 0.78 | +0.33 overconfident |
| OmniMath (advanced math) | 0.21 | 0.37 | 0.46 | +0.17 overconfident |
| SciCode (scientific code) | 0.44 | 0.62 | 0.59 | +0.18 overconfident |
In the full cloud
Llama 3.1 70B conditions all other model/condition points equal relative confidence and pass rate
Pairwise footprint
Match accuracy controls for the performance base-rate gap
Llama 3.1 70B pairs
18/ 171
Llama 3.1 70B mean tau
+0.050
All-pairs mean
+0.037
Llama 3.1 70B p<0.05
11(61.1%)
all model pairs (observed) base-rate-matched null calibration-preserving null Llama 3.1 70B pair (filled = p<0.05) Llama 3.1 70B mean all-pairs mean