PotARCin extends ARC with five-dimensional evaluation of abstract-reasoning skills
PotARCin is a new benchmark that extends the Abstraction and Reasoning Corpus (ARC) by testing understanding of a task's underlying rule across five dimensions—Definition, Classification, Constrained Generation, Editing, and Inversion—using programmatic instance generation. Evaluated on the ARC-AGI-1 training set, five modern models show a 25–52 percentage-point drop relative to standard ARC accuracy, and a held-out P-ARC test set yields 1–8% accuracy across the five dimensions, with models often contradicting their own stated rules.