Benchmark · Reasoning
ARC-AGI-1 (Abstraction and Reasoning Corpus)
Few-shot abstract pattern induction on grid puzzles from a handful of input-output examples.
Definition
- Short name
- ARC-AGI-1
- Category
- Reasoning
- Introduced
- 2019
- Metric
- Task accuracy (%) on the semi-private or private set (pass@2), reported with cost per task
- Direction
- Higher is better
- Official site
- arcprize.org — official site of ARC-AGI-1 (Abstraction and Reasoning Corpus) (external site)
- Reference check
- Confirmed against the paper or official site
Health review
As of
Saturated
Top models cluster near the ceiling; small differences are noise.
High-compute results on the semi-private set reached 87.5% in December 2024.
Source: arcprize.org — source for this health review (external site)
Caveats
- o3-preview scored 75.7% (low compute) and 87.5% (high compute) on the semi-private set in December 2024; results depend heavily on compute spent.Source: arcprize.org — source for caveat 1 on ARC-AGI-1 (Abstraction and Reasoning Corpus) (external site)
- Only ARC Prize-run semi-private results are verified; public-set scores are self-reported.Source: arcprize.org — source for caveat 2 on ARC-AGI-1 (Abstraction and Reasoning Corpus) (external site)
Results on file
No results on file yet
No result for ARC-AGI-1 is on file yet. We list a result only when we can trace it to a published source, and record the settings exactly as that source states them. HARBAN has not yet published internal evaluations of this benchmark.