Evaluation
Benchmarks
- Benchmarks
- 14
- Results on file
- 35
- Last checked
How to read this catalogue
Provenance
Every number carries a label saying who produced it. Of the 35 results on file, 33 are vendor-reported, 2 are third-party. HARBAN has not yet published internal evaluations.
- Vendor-reportedPublished by the organisation that built the model. Not independently reproduced.
- Third-partyMeasured and published by an independent evaluator.
- HARBAN internalRun by HARBAN AI LABS with published settings and logs.
Health
Benchmarks age. Each one carries a dated health label — active, saturated, flawed or retired — backed by a public source, so you know how much weight a score can bear before you read it.
Comparability
Results are grouped by benchmark and variant — version, subset, tool setting — and are only charted against results from the same group. Vendor-reported numbers inside a group still use different settings.
Catalogue
Benchmarks in the Mathematics category (2)
| Benchmark | Category | Health | Year | Results on file | Metric |
|---|---|---|---|---|---|
| AIMEAmerican Invitational Mathematics Examination (AIME), as used in LLM evals | Mathematics | Saturated | Not verifiedReference unconfirmed | 3 | Accuracy (%), usually mean over k samples per problem |
| MATHMATH (Measuring Mathematical Problem Solving With the MATH Dataset) | Mathematics | Flawed | 2021 | None yet | Exact-match accuracy (%) of final answer |
Health review
A health label is a dated judgement, not a permanent verdict. Each benchmark page gives the note behind its label, the date it was reviewed and the public source it rests on.
- Active
- No documented saturation, major flaw or retirement found as of the review date.
- 4 benchmarks overall
- Saturated
- Top models cluster near the ceiling; small differences are noise.
- 4 benchmarks overall
- Flawed
- Documented problems — label errors, weak tests or contamination — materially affect scores.
- 6 benchmarks overall
- Retired
- No longer maintained or superseded by a successor.
- 0 benchmarks overall