GPT sits close to the field mean on performance z-score and slightly above the field mean on confidence z-score. The family profile spans a wider performance range than confidence range. The family mean sits slightly above the diagonal, indicating higher relative confidence than relative performance.
Where OpenAI · GPT sits in the cloud
Disposition: where each model draws the line
Competitive context, benchmark by benchmark
| # | Model | Fβ ? | Prec ? | Rec ? | Task acc ? |
|---|---|---|---|---|---|
| 1 | Gemini 3 Pro | 0.785 | 0.77 | 0.80 | 0.65 |
| 2 | Gemini 3 Flash | 0.782 | 0.70 | 0.88 | 0.59 |
| 3 | Gemini 2.5 Pro | 0.764 | 0.71 | 0.83 | 0.58 |
| 4 | Claude Sonnet 4.5 | 0.753 | 0.78 | 0.72 | 0.59 |
| 5 | GPT-4o | 0.739 | 0.69 | 0.80 | 0.54 |
| 6 | DeepSeek R1 | 0.733 | 0.76 | 0.70 | 0.57 |
| 7 | Claude Haiku 4.5 | 0.716 | 0.79 | 0.65 | 0.48 |
| 8 | Mistral Medium 3.1 | 0.703 | 0.65 | 0.77 | 0.52 |
| 9 | GPT-4o mini | 0.686 | 0.56 | 0.88 | 0.45 |
| 10 | Gemini 2.0 Flash | 0.683 | 0.58 | 0.82 | 0.45 |
| 11 | GPT-5.2 | 0.673 | 0.84 | 0.56 | 0.62 |
| 12 | Gemini 2.5 Flash | 0.670 | 0.58 | 0.79 | 0.45 |
| 13 | Mistral Small 3.2 | 0.666 | 0.57 | 0.79 | 0.44 |
| 14 | Llama 3.1 70B | 0.643 | 0.65 | 0.64 | 0.45 |
| 15 | Claude 3.5 Sonnet | 0.643 | 0.79 | 0.54 | 0.56 |
| 16 | Llama 3.3 | 0.593 | 0.64 | 0.55 | 0.45 |
| 17 | Qwen 2.5 72B | 0.572 | 0.74 | 0.47 | 0.47 |
| 18 | DeepSeek Chat | 0.567 | 0.78 | 0.45 | 0.55 |
| 19 | Claude 3 Haiku | 0.554 | 0.55 | 0.56 | 0.40 |
| 20 | Qwen 2.5 Coder 32B | 0.476 | 0.31 | 1.00 | 0.31 |
The four metacognitive outcomes — OpenAI · GPT
How many universities does Newcastle have?
It said it could answer: “I have knowledge of the universities located in Newcastle, which include Newcastle University and Northumbria University. Therefore, I can provide an accurate count of the universities in Newcastle.”
→ It and it did.It answered “Newcastle has two universities: Newcastle University and Northumbria University.”; expected “two”.
What's the nickname for Newcastle's A167?
It said it could answer: “The A167 in Newcastle is commonly known as the 'Central Motorway'. This is a well-known nickname for this road, and I can provide this information based on existing knowledge.”
→ It but it couldn't.It answered “The nickname for Newcastle's A167 is "The Central Motorway."”; expected “the old "Great North Road"”.
How many Catholic schools were in Victoria?
It said it couldn't answer: “The number of Catholic schools in Victoria depends on the timeframe (specific year), what counts as a Catholic school (primary/secondary, special schools, campuses), and the data source. Without that context, any number would be a guess.”
→ It but it could.It answered “There were 489 Catholic schools in Victoria.”; expected “489”.
Find the last 3 digits of 2003^(2002^2001).
It said it couldn't answer: “To find the last 3 digits of a number, we need to compute it modulo 1000. This involves using modular arithmetic techniques such as Euler's theorem or the Chinese Remainder Theorem. Given the large exponents, the problem can be simplified using properties of modular arithmetic and possibly the Carmichael function to reduce the exponent. However, without performing detailed calculations, it's not immediately clear if the solution can be found without error. Therefore, while I can attempt to solve it, the complexity suggests that having multiple-choice options would help verify the solution.”
→ It and it couldn't.It answered “947”; expected “241”.