Benchmark · Reasoning
ARC-AGI-2
Harder, human-calibrated abstract reasoning grid tasks (120 each in public/semi-private/private eval sets) solvable by at least two humans in two attempts.
Definition
- Short name
- ARC-AGI-2
- Category
- Reasoning
- Introduced
- 2025
- Metric
- pass@2 accuracy (%) with cost per task
- Direction
- Higher is better
- Reference check
- Confirmed against the paper or official site
Health review
As of
Active
No documented saturation, major flaw or retirement found as of the review date.
The ARC-AGI-2 page still poses reaching 85% accuracy as an open challenge and links an ARC-AGI-2 competition; no saturation or retirement notice found as of the review date.
Source: arcprize.org — source for this health review (external site)
Caveats
- Only scores ARC Prize runs itself on the Semi-Private set appear on its Verified Leaderboard; Community Leaderboard entries are not verified by default. Say which set and leaderboard a score comes from.Source: arcprize.org — source for caveat 1 on ARC-AGI-2 (external site)
- ARC Prize treats efficiency as part of the result: starting with ARC-AGI-2, all its reporting comes with a cost metric, so show cost alongside accuracy.Source: arcprize.org — source for caveat 2 on ARC-AGI-2 (external site)
Results on file
No results on file yet
No result for ARC-AGI-2 is on file yet. We list a result only when we can trace it to a published source, and record the settings exactly as that source states them. HARBAN has not yet published internal evaluations of this benchmark.