← Atlas home Company / family view 5 models in the bench

Google · Gemini

Same public data as the home page, specialized to one familyParametric view: company_family

Gemini has the highest family mean on the performance z-score axis and also sits high on the confidence z-score axis. The family mean lies close to the diagonal, with relative confidence and relative performance nearly matched. The highlighted cloud is concentrated in the upper-right region of the map.

Where Google · Gemini sits in the cloud

-3-3-2-2-1-100112233Performance z-score within benchmark/probe →Confidence z-score →
Google · Gemini model/condition points Google · Gemini model/condition points mean x/y all other model/condition points equal relative confidence and pass rate

Disposition: where each model draws the line

0.000.250.500.751.00Gemini 2.0 FlashGemini 2.5 FlashGemini 3 FlashGemini 2.5 ProGemini 3 Propass rateimplied confidence (red gap = overconfident)

Competitive context, benchmark by benchmark

SQuAD (factual recall) · Prospective · Fβ β
#ModelFβ ?Prec ?Rec ?Task acc ?
1 Gemini 3 Pro0.7850.770.800.65
2 Gemini 3 Flash0.7820.700.880.59
3 Gemini 2.5 Pro0.7640.710.830.58
4 Claude Sonnet 4.50.7530.780.720.59
5 GPT-4o0.7390.690.800.54
6 DeepSeek R10.7330.760.700.57
7 Claude Haiku 4.50.7160.790.650.48
8 Mistral Medium 3.10.7030.650.770.52
9 GPT-4o mini0.6860.560.880.45
10 Gemini 2.0 Flash0.6830.580.820.45
11 GPT-5.20.6730.840.560.62
12 Gemini 2.5 Flash0.6700.580.790.45
13 Mistral Small 3.20.6660.570.790.44
14 Llama 3.1 70B0.6430.650.640.45
15 Claude 3.5 Sonnet0.6430.790.540.56
16 Llama 3.30.5930.640.550.45
17 Qwen 2.5 72B0.5720.740.470.47
18 DeepSeek Chat0.5670.780.450.55
19 Claude 3 Haiku0.5540.550.560.40
20 Qwen 2.5 Coder 32B0.4760.311.000.31

The four metacognitive outcomes — Google · Gemini

Competent claimed it could — and could
Gemini 2.5 Pro · SQuAD

Who was the ABC Radio president in 1968?

It said it could answer: “The question is a factual query about a specific historical role within a major corporation. This information is typically well-documented in public records, news archives, and corporate histories that are part of my knowledge base.”

→ It and it did.It answered “Harold L. Neal Jr.”; expected “Harold L. Neal”.

Overconfident claimed it could — but couldn’t
Gemini 2.5 Pro · SQuAD

What was ABC's viewership share in 1993?

It said it could answer: “This is a factual question about historical television ratings. My training data includes extensive historical information, including data on network viewership shares from sources like Nielsen ratings, which would cover the year 1993.”

→ It but it couldn't.It answered “During the 1992–93 and 1993–94 U.S. television seasons, ABC's average viewership share was 21%.”; expected “23.63% of American households”.

Underconfident declined — but could
Gemini 2.0 Flash · SQuAD

About many students attend Kunskapsskolan schools?

It said it couldn't answer: “While I know Kunskapsskolan is a school system, I do not have access to real-time data or specific enrollment numbers for their schools. Therefore, I cannot provide an accurate answer to the question about the number of students attending.”

→ It but it could.It answered “Around 10,000.”; expected “10,000”.

Well-declined declined — and couldn’t
Gemini 2.5 Flash · SQuAD

How much of the Rhine flow does Ijssel carry?

It said it couldn't answer: “The question asks for a specific numerical value (how much of the Rhine flow), which is factual information not inherently known or derivable without external data. I would need access to current or historical hydrological data for the Rhine and Ijssel rivers to provide an accurate answer.”

→ It and it couldn't.It answered “11%”; expected “one ninth”.

Rows: top = claimed it could answer · bottom = declined Columns: left = was actually correct · right = was actually wrong

Pairwise signal: pairs involving Google · Gemini

Match accuracy controls for the performance base-rate gap
Google · Gemini pairs
80/ 171
Google · Gemini mean tau
+0.052
All-pairs mean
+0.037
Google · Gemini p<0.05
48(60%)
-1.0-0.50.00.51.0Pair signal: do confidence gaps rank performance gaps? (Kendall tau-b)
all model pairs (observed) base-rate-matched null calibration-preserving null Google · Gemini pair (filled = p<0.05) Google · Gemini mean all-pairs mean
Gemini 2.0 Flash →Gemini 2.5 Flash →Gemini 2.5 Pro →Gemini 3 Flash →Gemini 3 Pro → Compare against peers →