Benchmark · Mathematics
American Invitational Mathematics Examination (AIME), as used in LLM evals
Definition
- Short name
- AIME
- Category
- Mathematics
- Introduced
- Not verifiedReference not confirmed — see below
- Metric
- Accuracy (%), usually mean over k samples per problem
- Direction
- Higher is better
- Paper
- No paper
- Official site
- maa.org — official site of American Invitational Mathematics Examination (AIME), as used in LLM evals (external site)
- Reference check
- Not confirmed — at least one reference detail (such as the year or canonical page) could not be verified
Health review
As of
Top models cluster near the ceiling; small differences are noise.
The leading models on Artificial Analysis’s AIME 2025 leaderboard score at or near 100%, and Artificial Analysis now lists it as a legacy evaluation.
Source: artificialanalysis.ai — source for this health review (external site)
Caveats
- Contamination: MathArena reported strong signs of contamination in AIME 2024.Source: arxiv.org — source for caveat 1 on American Invitational Mathematics Examination (AIME), as used in LLM evals (external site)
- Artificial Analysis removed AIME 2025 from its Intelligence Index in v4.0 (January 2026), without stating a reason, and has retired it from active reporting.Source: artificialanalysis.ai — source for caveat 2 on American Invitational Mathematics Examination (AIME), as used in LLM evals (external site)
- Small set: AIME 2025 has 30 problems in total, so one problem is worth about 3.3 percentage points and a single run is noisy.Source: artificialanalysis.ai — source for caveat 3 on American Invitational Mathematics Examination (AIME), as used in LLM evals (external site)
Results on file
3 results in 2 comparability groups (3 vendor-reported). Results are only charted against others in the same group — the same benchmark and the same variant as named by the source. Last retrieved .
AIME · 2026
2 results
Not directly comparable
These are vendor-reported numbers from 2 different organisations. Each source chose its own settings — reasoning effort, tools, harness or scaffold, number of attempts, grader — so a longer bar is a claim made under different conditions, not evidence that one model is better. None of these results has been reproduced by HARBAN.
| Model | Model detail | Score | Settings (as stated) | Source | Evidence |
|---|---|---|---|---|---|
| Muse Glimmer | High Reasoning | 94.7%No CI reported | High Reasoning | Show excerpt for Muse Glimmer (High Reasoning) on AIME, 2026“| AIME 2026 | **94.7** | 89.2 | 94.1 |” Retrieved | |
| Gemma 4 31B | Not stated by source | 89.2%No CI reported | no tools; instruction-tuned model | Show excerpt for Gemma 4 31B on AIME, 2026“| AIME 2026 no tools | 89.2% |” Retrieved |
- Vendor-reported:
- Published by the organisation that built the model. Not independently reproduced.
AIME · 2025
1 result
Not directly comparable
These are vendor-reported numbers. Each source chose its own settings — reasoning effort, tools, harness or scaffold, number of attempts, grader — so a longer bar is a claim made under different conditions, not evidence that one model is better. None of these results has been reproduced by HARBAN.
Only one result is on file for this variant, so there is nothing to chart it against.
| Model | Model detail | Score | Settings (as stated) | Source | Evidence |
|---|---|---|---|---|---|
| gpt-oss-120b | reasoning level high | 92.5%No CI reported | no tools; reasoning level high (Table 3) | Show excerpt for gpt-oss-120b (reasoning level high) on AIME, 2025“AIME 2025 (no tools) 50.4 80.0 92.5” Retrieved |
- Vendor-reported:
- Published by the organisation that built the model. Not independently reproduced.