Qwen has the lowest family mean on the performance z-score axis while sitting near the field mean on confidence z-score. The family has the largest confidence-over-performance separation in this view. The highlighted cloud sits above the diagonal at the family mean.
Where Qwen sits in the cloud
Disposition: where each model draws the line
Competitive context, benchmark by benchmark
| # | Model | Fβ ? | Prec ? | Rec ? | Task acc ? |
|---|---|---|---|---|---|
| 1 | Gemini 3 Pro | 0.785 | 0.77 | 0.80 | 0.65 |
| 2 | Gemini 3 Flash | 0.782 | 0.70 | 0.88 | 0.59 |
| 3 | Gemini 2.5 Pro | 0.764 | 0.71 | 0.83 | 0.58 |
| 4 | Claude Sonnet 4.5 | 0.753 | 0.78 | 0.72 | 0.59 |
| 5 | GPT-4o | 0.739 | 0.69 | 0.80 | 0.54 |
| 6 | DeepSeek R1 | 0.733 | 0.76 | 0.70 | 0.57 |
| 7 | Claude Haiku 4.5 | 0.716 | 0.79 | 0.65 | 0.48 |
| 8 | Mistral Medium 3.1 | 0.703 | 0.65 | 0.77 | 0.52 |
| 9 | GPT-4o mini | 0.686 | 0.56 | 0.88 | 0.45 |
| 10 | Gemini 2.0 Flash | 0.683 | 0.58 | 0.82 | 0.45 |
| 11 | GPT-5.2 | 0.673 | 0.84 | 0.56 | 0.62 |
| 12 | Gemini 2.5 Flash | 0.670 | 0.58 | 0.79 | 0.45 |
| 13 | Mistral Small 3.2 | 0.666 | 0.57 | 0.79 | 0.44 |
| 14 | Llama 3.1 70B | 0.643 | 0.65 | 0.64 | 0.45 |
| 15 | Claude 3.5 Sonnet | 0.643 | 0.79 | 0.54 | 0.56 |
| 16 | Llama 3.3 | 0.593 | 0.64 | 0.55 | 0.45 |
| 17 | Qwen 2.5 72B | 0.572 | 0.74 | 0.47 | 0.47 |
| 18 | DeepSeek Chat | 0.567 | 0.78 | 0.45 | 0.55 |
| 19 | Claude 3 Haiku | 0.554 | 0.55 | 0.56 | 0.40 |
| 20 | Qwen 2.5 Coder 32B | 0.476 | 0.31 | 1.00 | 0.31 |
The four metacognitive outcomes — Qwen
How often does the General Conference meet?
It said it could answer: “The General Conference, in the context of the United Methodist Church, meets every four years. This is a well-established fact within the structure of the church and does not require additional information to answer accurately.”
→ It and it did.It answered “The General Conference meets every four years.”; expected “every four years”.
Who was the 4th to rule the Yuan dynasty?
It said it could answer: “As a knowledgeable assistant, I am aware of the succession of Yuan dynasty emperors, and I can provide the name of the 4th ruler without additional information.”
→ It but it couldn't.It answered “The 4th ruler of the Yuan dynasty was Emperor Wenzong, also known as Tugh Temur.”; expected “Buyantu Khan”.
How long had John Paul II been the pope in 1979?
It said it couldn't answer: “John Paul II was elected Pope in October 1978, so to determine how long he had been Pope in 1979, we would need to calculate the exact duration from his election date to a specific point in 1979. Without specifying a particular date in 1979, it is not possible to provide an exact answer.”
→ It but it could.It answered “John Paul II had been the pope for about 5 months in 1979.”; expected “less than a year”.
Who were the announcers of Super Bowl 50?
It said it couldn't answer: “I do not have the specific information about the announcers for Super Bowl 50 memorized, and without looking up the details, I cannot provide an accurate answer.”
→ It and it couldn't.It answered “The announcers for Super Bowl 50 were Joe Buck and Troy Aikman.”; expected “Jim Nantz and Phil Simms”.