The papers behind ARC-AGI — and the agentic turn.

By 2019 nearly every AI benchmark rewarded skill at one task — playing Go, classifying images, answering trivia — and Chollet's central move is to insist that skill is not intelligence. Skill is the product of intelligence and prior knowledge plus experience, so a system with enough training data, compute, or hand-coded priors can reach superhuman performance on almost any single task without showing any general capacity to learn. Measuring intelligence by task score therefore lets researchers "buy" skill and systematically confuse the map (a benchmark number) with the territory (the capacity to acquire new skills).

The constructive core is a formal, information-theoretic definition grounded in Algorithmic Information Theory: intelligence is skill-acquisition efficiency over a scope of tasks, controlling for priors, experience, and generalization difficulty. A system needing a million examples is less intelligent than one needing three, even if both eventually reach the same skill. Chollet fixes fair priors as Core Knowledge — the innate systems developmental psychology attributes to humans: objectness and elementary physics, agentness and goals, number and arithmetic, geometry and topology. He operationalizes the measure with the Abstraction and Reasoning Corpus: about 1,000 hand-built visual puzzles, each a few input/output grid pairs (up to 30×30, small palette). From roughly three-to-five demonstrations the solver must infer the transformation rule and apply it to a held-out input, with a private test set for honest scoring.

The paper is a measurement contribution, not a solver: its result is the definition, the benchmark, and a quantified gap — humans solved the large majority of evaluation tasks while the best automated methods solved only a small fraction. The bet proved durable; ARC resisted the pretraining-scaling era far longer than task-specific benchmarks, exactly as the theory predicted. It reframed "AGI" from a vague aspiration into a measurable gap between human and machine generalization, and defined the yardstick that ARC-AGI-2, ARC-AGI-3, and the ARC Prize have extended rather than replaced.

In late 2024 OpenAI's o3, spending large amounts of test-time compute, effectively solved the original ARC-AGI-1 benchmark — high-70s to high-80s on the semi-private set, at a cost between roughly \$200 and \$20,000 per task. Rather than close the challenge, the authors argue the result exposed where ARC-AGI-1 had become saturated and un-granular near the top, not that the human-machine gap had closed. ARC-AGI-2 is a re-designed, harder benchmark that restores a strong, measurable gap and makes efficiency a first-class axis.

The method is human-in-the-loop and adversarial toward frontier models. The paper deliberately preserves the grid-pair format so existing tooling carries over, then curates tasks targeting four demands that defeated o3: multi-rule composition, sequential/multi-step reasoning, context-dependent control flow, and in-context symbol definition. A pool of a couple thousand candidate pairs is put through live human testing; tasks too few humans can solve are discarded, guaranteeing every evaluation task is solvable by at least two independent humans, while tasks models find easy give little signal. A private/semi-private split blocks tuning, and the leaderboard reports each system as a point in (cost-per-task, score) space.

The gap reopens sharply. On the semi-private set o3 (medium) and o3-mini (high) land around 3%, the 2024 ARChitects winner around 2.5%, and o1-pro (low) and Claude 3.7 around 0.9% — versus 53%, 34.5%, 23.3%, and 21.2% on ARC-AGI-1; scores below ~5% aren't treated as signal. The offline, fixed-compute ARC Prize 2025 competition set an 85% grand-prize bar, drew ~1,455 teams, and saw a best entry near 24% using synthetic-data generation plus test-time training on a ~4B model — the bar went unclaimed. The lesson — measure intelligence jointly with efficiency — is carried directly into ARC-AGI-3.

ARC-AGI-1 and ARC-AGI-2 tested data-efficient modeling: infer a transformation rule from a handful of static input/output pairs. But static benchmarks can be attacked by training on dense samplings of the task space. ARC-AGI-3 keeps the Core Knowledge, no-language, novelty-first spirit while changing the format entirely: the agent is dropped into abstract, turn-based game environments and told nothing — not the goal, not the controls, not the win condition. It must actively discover even what question it is being asked, targeting the "unknown unknowns."

The benchmark decomposes agentic intelligence into four components that must all be exercised: Exploration, Modeling, Goal-Setting, and Planning & Execution. The environment format is deliberately minimal so difficulty lives in reasoning, not perception: each frame is a 64×64 grid of 16 colors, the world is turn-based and never mutates asynchronously, and the action space is tiny (five keys + Undo + click-a-cell), with only state-changing actions counting. All four pillars collapse into a single scalar, RHAE (Relative Human Action Efficiency): per level, the human baseline action count divided by the agent's, capped at 1.15×, then squared as a power-law penalty on inefficiency, level-weighted, capped by the fraction of levels completed, and averaged over ~135 environments. A studio validates each environment with random-play qualification, a graph of hashed reachable states with per-level win probability, and recording-based replay verification.

After human calibration (humans reach 100%), frontier models score below 1% — roughly 0.1–0.5% harness-free at release; the best preview agent, "StochasticGoose," reached ~12.58% via informed search rather than genuine modeling. The authors argue this is a real deficit in exploration, hypothesis revision, goal inference, and planning under uncertainty, not an API artifact. By grounding the score in first-contact action efficiency, ARC-AGI-3 turns "acquire a correct world model quickly" into a measurable quantity, presenting itself as the only unsaturated general agentic benchmark as of March 2026.

ARC-AGI-1 had gone about five years essentially unbeaten, and the 2024 ARC Prize was the first large public push against it. The question the season answered: what actually moves the needle on genuinely novel tasks — scaling a single model, or something else? The report reframes ARC solving around test-time adaptation, arguing that pure "memorize-fetch-apply" inference is not enough.

The report is a survey of competitor methods organized by the transduction-vs-induction distinction. Induction searches for an explicit program — a hand-built DSL or Python emitted by an LLM — keeping only programs that exactly reproduce every training pair, then runs the survivors on the held-out test input; LLM-guided variants propose likely-correct programs and debug failures to shrink the search. Transduction predicts the output grid directly, typically via test-time training: for each task, LoRA fine-tune a pretrained LLM on that task's demonstration pairs plus heavy augmentation (rotations, reflections, color permutations), then vote among augmentation-consistent candidate outputs. The key empirical finding is that these two strategies solve substantially different task sets, so the best systems run both pipelines and reconcile their outputs.

Private-set state of the art rose from ~33% (end of 2023) to 55.5% over the roughly five-month season. The ARChitects won Kaggle at 53.5% using TTT, augmentation, and a product-of-experts selection criterion; MindsAI hit the season-high 55.5% but stayed closed and was prize-ineligible. Greenblatt's GPT-4o synthesis reached ~42% by sampling and debugging thousands of Python programs per task, and Akyürek et al.'s TTT paper hit ~47.5% and won a paper award. Crucially, no static, non-adapting approach exceeded ~11%. But as a brute-force synthetic-data ensemble reached the low 80s, the report candidly judged ARC-AGI-1 near saturation and attackable — directly motivating the cost-aware ARC-AGI-2 and interactive ARC-AGI-3.

Where o3 saturated ARC-AGI-1 in 2024, ARC-AGI-2 reopened a wide human-machine gap, and the 2025 competition was the first run against it. This report documents the result: participation surged (~1,455 teams, ~15,000 entries, paper submissions nearly doubling to ~90) but scores stayed low. The top private-leaderboard entry reached only ~24% against the 85% grand-prize threshold, which went unclaimed — and it got there cheaply, ~$0.20 per task, using synthetic data plus test-time training on a compact few-billion-parameter model. Humans still solve essentially all evaluation tasks, so the gap is stark.

The report's central diagnosis is the emergence of the refinement loop: a per-task, feedback-guided optimization cycle. Rather than sampling candidates once, the strongest entries propose a solution (a program, a chain-of-thought, or a set of weights), verify it against the training pairs, convert the discrepancy into feedback, and iterate until it fits every demonstration. It taxonomizes three flavors: evolutionary program synthesis (populations refined against a verification fitness, e.g. the Berman and Pang entries); application-layer refinement of commercial models (a verifier wraps a frontier model's chain-of-thought — one Gemini-3-Pro-class harness reportedly climbed from ~31% to ~54% at ~$31/task); and weight-space/zero-pretraining refinement, exemplified by the Tiny Recursive Model (TRM): ~7M parameters, no pretraining, ~45% on ARC-AGI-1 but only ~8% on ARC-AGI-2.

Two further themes give the report weight. It introduces knowledge-dependent overfitting as a subtler contamination category: performance is bounded by knowledge coverage, and ARC's own conventions may now be latent in base models — the report notes a model applying the correct ARC color mapping without ever being told the task was ARC. And it marks ARC-AGI as an industry standard (Anthropic, Google DeepMind, OpenAI, and xAI all reported numbers in 2025 model cards) before previewing ARC-AGI-3, the first interactive format change since 2019.

For four years, ARC-AGI-1 — puzzles designed to be easy for humans and hard for machines — resisted large language models. Scores crept from ~0% (GPT-3, 2020) to only ~5% (GPT-4o, 2024). Standard models operate by "memorize, fetch, apply": they retrieve a memorized transformation and apply it, which fails on genuinely novel tasks. The open question was whether frontier models could ever adapt to the unseen.

o3 changed the mechanism rather than the scale. At test time it generates many candidate chains-of-thought — natural-language programs for the specific task — and uses a learned evaluator to search and select among them, an AlphaZero-like generate-and-score loop rather than a single forward pass. Chollet calls this deep learning-guided program search, with the chain-of-thought itself as the artifact of knowledge recombination. The "high" versus "low" efficiency settings correspond to how much test-time search compute is spent; the low-efficiency run used ~172× the compute. ARC Prize evaluated under retail-cost accounting that makes per-task price visible.

o3 scored 75.7% (high efficiency, ~$20/task) and 87.5% (172× compute) on the 100-task semi-private set, and ~82.8% / ~91.5% on the 400-task public set — clearing o1's 32% ceiling and the 53% best prior hand-built method. But the caveats made it a pivotal document: cost ran into the thousands per task against ~$5 for a human, o3 still fails some very easy tasks, and early ARC-AGI-2 testing suggested it would drop under ~30% while humans stay above ~95%. Combined with a ~81% Kaggle ensemble, the result signaled ARC-AGI-1 was saturating and motivated the harder ARC-AGI-2 and the interactive ARC-AGI-3 — plus a restated open-source 85%-at-efficiency prize bar that o3 did not clear.

Each ARC task is a handful of input to output grid pairs plus a test input; grids are small (up to ~30×30) with 10 colors, and every task is novel — you cannot memorize, you must infer the transformation from a few demonstrations. In 2020 the open question was how far pure symbolic search could get with no learning at all, and this solution is the archetypal answer.

The solver represents grids as "images" and maintains, per task, a growing pool of candidate pieces. It seeds the pool with the input(s), then up to a bounded depth applies its library of ~142 unary functions to existing pieces, adding each result — a greedy DAG expansion (a DAG rather than a tree because one piece feeds many functions). After building the pool it searches for a stacking or composition of pieces that exactly matches the training outputs across all pairs; any program consistent with every pair is executed on the test input. Depth is the key knob: depth 2 runs the public eval in ~70 seconds but is limited, while depth 3 with test prediction is far stronger and takes ~9 hours on a single 4-core machine. The core is written in C++ for speed with Python orchestration, running fully offline within Kaggle's compute limits.

It scored ~20% on the private leaderboard for first place as a solo competitor, and ~129/419 tasks at depth 2 on the public evaluation set. Historically it is a landmark: it established the canonical ARC-AGI-1 style — hand-built DSL, exhaustive/greedy compositional search, exact-match-on-training-pairs verifier — that dominated until 2024's test-time training and LLM-guided synthesis. Its ceiling, a solo human authoring primitives and a search that explodes combinatorially with depth, is exactly the limitation learned DSLs, neural-guided search, and LLM program generation set out to break.

In-context learning plateaus badly on genuinely out-of-distribution reasoning like ARC: a model can see a few input/output grid pairs but cannot reliably induce the underlying transformation. This paper asks whether test-time training — briefly updating the model's own parameters at inference from a self-supervised loss on the task's own data, then throwing those weights away — can close the gap between what an LM knows and what it can actually do on structurally novel problems.

For each test task the system builds a per-task training set purely from the demonstration pairs using a leave-one-out scheme: hold one pair out as the query while the rest serve as in-context support, then multiply by invertible augmentations (D8 rotations, reflections, transpose, plus color permutations). A fresh LoRA adapter is trained on this synthetic set for a few steps and discarded afterward. It all sits atop a base model first fine-tuned on ARC-like data (ReARC/BARC-style generators). At decode time the input is passed through the same invertible transforms, predictions are un-transformed back to the canonical frame, and a two-stage hierarchical self-consistency vote picks the answer. Ablations isolate three necessary ingredients — initial fine-tuning, the leave-one-out augmented format, and per-instance adapters — each responsible for a measurable chunk of accuracy.

The payoff: an 8B LM reaches 53.0% on ARC public validation with TTT, up to 6x over a fine-tuned baseline, and 61.9% when ensembled with the BARC program synthesizer — matching reported average human performance (~60.2%). The recipe transfers, beating 10-shot prompting on BIG-Bench Hard by 7.3 points (50.5% → 57.8%). The broader message challenges the frozen-weights-plus-long-context orthodoxy: weight adaptation at test time is a distinct axis of test-time compute. A caution the note flags is that these ARC-AGI-1 gains collapse to single digits on ARC-AGI-2, so adaptation must be paired with strict verification.

ARC solving in 2024 faced a hard practical question that headline scores obscured: what wins under the competition's real constraints — fully offline, fixed consumer hardware, bounded time, and roughly two cents per task? The ARChitects' answer is that the decisive lever is not a bigger model or more search, but perspective. The same task, viewed under many equivalent transformations, gives a model many partially-independent readings, and combining those readings correctly is what separates a mediocre single-shot solver from a state-of-the-art one.

The pipeline starts from an open Mistral-NeMo-Minitron-8B base, fine-tuned on augmented ARC-style data, then applies test-time training: for each evaluation task a fresh rank-32 LoRA adapter is fit in about 51 seconds on an RTX 4090, specializing the model to that task's transformation — the single largest accuracy jump, as in the MIT TTT work. Around it sit two engineered pieces. Candidate generation replaces wasteful stochastic sampling with a depth-first token sampler that expands the output grid and prunes any branch whose cumulative probability drops below a threshold T, emitting only about nine candidates per task. Scoring is the named idea: each candidate is transformed into many invertible perspectives, scored under each, and the probabilities multiplied (a product of experts), with a theoretical bound relating the ensemble's divergence to the average single-view divergence plus a gain when views disagree.

The result: 1st place at 53.5% on the private set, and 71.6% on ARC-AGI-1 public eval (about 286/400) with the post-competition pipeline — above the ~60% average-human mark and state of the art among open methods, though below concurrent o3 (~82.8%) at far higher cost. A clean ablation ladder (baseline → TTT → multi-augmentation inference → PoE → DFS) quantifies each step, and ~73% on ConceptARC argues the gains are not pure in-distribution memorization. It is the strongest public evidence that augmentation-as-ensembling, not scale, was 2024's highest-leverage idea.

By mid-2024, ARC-AGI still stumped frontier LLMs, and a common framing held that models like GPT-4o simply could not synthesize genuinely novel solutions. A second, more practical obstacle sat underneath: GPT-4o's grid vision is poor — it reliably misreads grids larger than about 12x12 and struggles even at 8x8 — so any approach has to fight the model's perception, not just its reasoning.

Greenblatt's recipe sidesteps both with brute-force sampling under an execution filter. Each task is packed into a ~30k-token few-shot prompt that renders the training pairs in several redundant encodings at once — 2D arrays with spreadsheet-style cell coordinates, connected-component object listings, normalized shape descriptions, and cell-level diffs — with two prompt families for size-preserving versus size-changing tasks. GPT-4o then emits roughly 8,000 candidate Python transform functions; each is executed against every training pair and discarded unless it reproduces all of them exactly. A revision stage takes the dozen most promising failures, shows the model their concrete wrong outputs beside the expected ones, and draws ~3,000 repair samples. Surviving programs run on the test input, and the top-3 by majority vote are submitted.

The result was ~50% on a 100-task public-eval sample and ~42% on the compute-limited ARC-AGI-Pub set — state-of-the-art at the time, up from the low-30s, with no fine-tuning. Revision alone contributed ~13 points, a far cheaper path than blind resampling (reaching 50% from first-attempt samples would need on the order of 100,000 draws). Accuracy scales at roughly +3% per doubling of samples, so pushing to ~70% would demand millions of draws — the method costs about 1000x more test-time compute than prior work and is ineligible for the official prize. But the lesson stuck: GPT-4o's program distribution places real probability mass near correct solutions, which is exactly why filtered sampling works where random search would not.

Greenblatt had shown that sampling thousands of Python programs and filtering by execution reaches ~43% on ARC-AGI-Pub, but pure first-shot sampling plateaus: near-correct programs that each solve a different subset of examples get greedily discarded. Berman's question is whether *iterated refinement* — treating candidate programs as an evolving population — buys more than simply drawing more one-shot samples, all within the leaderboard's ~12-hour window and ~$10,000 API budget.

The method is inference-time only; the base model is never fine-tuned. For each task, Claude 3.5 Sonnet emits an initial generation of ~250 Python transforms from chain-of-thought one-shot prompts. Each is executed against the training pairs and scored by a two-tier fitness — how many grids are solved exactly, then per-cell correctness as a partial-credit tiebreaker. The fittest become parents for dynamically built revision prompts that breed improved children, repeating for up to four generations (~500 functions, ~31 prompts per task). A second, "pooled" track combines functions that solve *disjoint* example subsets into one revision prompt so their partial solutions can recombine — a diversity-preservation mechanism against premature convergence. Prompting mattered: CoT was required, a single well-chosen one-shot example beat multi-shot, and each grid was shown redundantly as dimensions, a base64 image, ASCII, and nested lists.

The payoff was a record 53.6% (268/500), roughly +10.6 points over Greenblatt's ~43% and unbeaten until o3. Crucially, depth beat breadth: four generations of 50 reached ~75% versus ~70% for a single generation of 200 on 60 training tasks, with ~42% of wins emerging in generations 2–4 — evidence that refinement, not just more samples, does real work. Berman closes with theory: LLMs fundamentally do induction, not deduction, and the "verifier problem" may cap any filtered evolution below AGI. A 2025 follow-up evolves natural-language solution descriptions instead of Python.

The ARC-solving field had split into two camps: inductive program synthesis (search for a rule f that reproduces the demos, then apply it to the test input) versus transduction (predict the held-out output directly with a neural network, no program ever materialized). Each side claimed superiority. This paper asks the cleanest possible version of the question — hold architecture, training data, and base model fixed, and let only the strategy differ — then compares the two head-to-head on ARC.

Both models are fine-tuned from Llama-3.1-8B-Instruct, with grids serialized one token per pixel. To build the training corpus, the authors hand-write ~100–160 "seeds," each a natural-language description, a transform_grid function (the latent f), and a generate_input sampler. An LLM then remixes them in three stages: recombine descriptions self-instruct style, generate code by retrieval over similar seeds, and execute the samplers to materialize fresh input/output grids — expanding 160 seeds into 200k tasks (ARC-Heavy) and up to 400k (ARC-Potpourri). At test time, induction samples up to ~20k candidate programs, keeps those consistent with the training pairs, and applies a survivor; transduction decodes by beam search. The ensemble tries induction — self-verifying against the demos — and falls back to transduction only when no consistent program is found.

The two strategies fail on genuinely different tasks (a stable Venn split of ~26 induction-only, ~35 transduction-only, ~19 solved by both; p < .004), so combining them with test-time training and reranking reaches 56.75% on ARC public validation — surpassing prior published methods and nearing average-human ~60.2% (best human ~97.8%). Induction excels at precise computation and composing concepts; transduction at perceptual judgments, corroborated on ConceptARC. Performance saturates with more human seeds but keeps scaling with synthetic data and compute; a later Akyürek et al. follow-up reused these BARC models to reach 61.9%.

Symbolic program synthesis searches a hand-designed domain-specific language; neural methods lean on heavy stochastic sampling or expensive per-task weight fine-tuning. Both give something up: the DSL is brittle and human-authored, while fine-tuning is costly and never structures a real adaptation mechanism. LPN asks whether test-time search can instead be learned and made part of the architecture, dropping the DSL entirely.

LPN uses two transformers. An encoder maps an input-output pair (x, y) to a distribution over a continuous latent z; a decoder maps a latent and a new input to an output, decoded pixel by pixel. Each z names an implicit program. Training is a leave-one-out variational scheme: for each pair, the encoder sees all the other pairs, and the decoder must reconstruct the held-out output — preventing the encoder from copying the target and forcing the latent to capture a reusable rule. Crucially, in the "Grad k" regime the model is trained with k gradient-refinement steps applied to the latent before decoding, so it learns a landscape actually navigable by gradient ascent. At test time you initialize z from the encoder, take gradient-ascent steps to maximize the log-likelihood of the training pairs, and decode the test input with the refined latent.

Evaluated on programming-by-example tasks and on ARC-AGI (a 178M-parameter model with a 256-dimensional latent, trained on ~51M RE-ARC pairs), LPN matches or beats in-context learning and test-time-training baselines — 78.5% in-distribution vs 57% for TTT — and its out-of-distribution ARC accuracy roughly doubles, from ~7.75% to ~15.5%, once test-time gradient search is enabled. Absolute OOD numbers stay low, but the headline is architectural: building search into the model gives structured, cheap per-task adaptation without a DSL or fine-tuning.

Inductive reasoning means inferring a general transformation rule from a handful of input/output examples and applying it to a novel input. The paper argues that the dominant way LLMs are evaluated on such tasks — directly prompting for the answer, i.e. in-context learning — is the wrong altitude: it works on easy inductive tasks but collapses on structurally novel benchmarks like ARC. Drawing explicitly on Bayesian models of human inductive learning (Tenenbaum, Goodman), the authors argue reasoning should proceed through an explicit hypothesis space — propose candidate rules, then test them — rather than interpolating an answer in one shot.

The method decomposes the problem into two levels of abstraction, run as Generate → Select → Implement → Validate. GPT-4 (gpt-4-0314, temperature 1.0) is prompted with the examples — for ARC, grids serialized as NumPy-style arrays with a color legend — and proposes ~64 natural-language hypotheses. Natural language captures a rule's gist but is ambiguous and un-executable, so each surviving hypothesis is implemented as a concrete Python program that runs against the training pairs; a program reproducing all outputs is accepted and applied to the test input, with up to 3 rounds of execution-feedback repair otherwise. The candidate set is pruned by LLM summarization (64→~8) or human selection. A key empirical finding: a program passing all training pairs almost always generalizes to the test item, making training verification a near-reliable acceptance gate — so the bottleneck is proposing a correct hypothesis at all (GPT-4 found one for only 49 of 100 ARC tasks).

On a 100-task ARC subset, accuracy climbs from 17% (direct) to 23% (program-only) to 30% (summarized) to 33% (human-selected), approaching Icecuber's 43% DSL-based SOTA without any ARC DSL. The gains hold cross-domain — 1D-ARC 73.1%, SyGuS strings 94.3% (beating DSL-based CrossBeam's 74.8%), List Functions 69%. The framing reframes an LLM not as an answer oracle but as a proposal distribution over programs, with an external executor supplying ground-truth reward — precisely the shape of a synthesize-then-verify loop.

ARC framed as programming-by-examples is brutal for a learning agent: given a handful of input/output grid pairs, produce a DSL program that maps one to the other. A code language model sampling programs essentially never hits the exact target output by chance, so ordinary reinforcement or rejection sampling collects a zero reward on almost every attempt and has nothing to learn from. CodeIt's insight is that a failed program is still a correct program for some task — just not the one you asked about.

The method is a self-improving loop. In each meta-iteration the 220M-parameter CodeT5+ policy samples up to 24 programs per task; each is run through the DSL interpreter, and every syntactically valid one is hindsight-relabeled — its realized output grids become the goal — and written to a replay buffer. The policy then fine-tunes with a negative-log-likelihood objective on a mixture of buffer experiences, mutation-augmented programs, and the original ground-truth seeds. Prioritized sampling up-weights experiences whose relabeled goal matches a real task demonstration, keeping learning anchored to genuine ARC transformations and fighting catastrophic forgetting. Grids are fed as sparse object-centric text (coordinates plus colors) rather than dense pixels to keep sequences short.

Run for 100 meta-iterations, CodeIt solves 59 of 400 ARC evaluation tasks (~15%) — reported as SOTA among methods scaled to the full evaluation set at the time, beating symbolic search baselines (26/400, 23/400) and a GPT-4 setup (49/412). Ablations show the loop genuinely depends on its parts: removing hindsight relabeling stalls learning at 24/400, removing pretraining collapses it to 9/400. CodeIt is not tabula-rasa learning — it leans on a hand-built DSL, an interpreter, and a seed set of programs — but it shows how to squeeze a dense, self-improving signal out of sparse-reward program search once that scaffolding exists.

The dominant ARC recipes all lean on some form of scale — pretraining, big synthetic corpora, exhaustive program search, or heavy test-time compute. CompressARC makes a deliberately extreme counter-claim: strip every one of those crutches away and puzzle-solving competence still emerges, purely from an inference-time compression objective. There is no training corpus and no fine-tuning phase anywhere in the loop.

For each individual puzzle, a fresh, randomly-initialized ~76K-parameter network is trained from scratch at inference time on that one puzzle alone, with the answer grid removed. The engine is the Minimum Description Length principle, operationalized as a VAE-style objective: gradient descent jointly optimizes a latent code and the decoder to minimize the total bits needed to describe the puzzle — the KL divergence of the latent against a standard-normal prior (coding the latent) plus cross-entropy reconstruction over the known grids. The missing answer is whatever the most-compressed model reconstructs in the blank cells; by Occam's razor the shortest description tends to be right. The priors live in an equivariant "multitensor" architecture whose weight tying bakes in ARC's symmetries (example order, color, rotation/reflection) so the net cannot cheat by memorizing orientations. Per puzzle it runs ~2000 Adam steps (~20 min on an RTX 4070); the reported answer is the most frequently sampled output (an EMA vote), with two guesses submitted (pass@2).

It reaches ~34% pass@2 on ARC-AGI-1's training set and ~20% on the eval set — versus a same-setup U-Net near 0.75%, with HRM at 40.3% (which trains on train+test) and OpenAI o3 at 87.5% (internet-scale) cited for calibration. On the harder ARC-AGI-2 a purely local, count-incapable method scores only low single digits (~4%). Empirically, lower-bit solutions correlate with more accurate ones, so compression doubles as a confidence signal. The published capability boundary — strong at local color/infill/line reasoning, helpless at counting, long-range structure, and any dynamics — makes "compression alone" unusually legible.

ARC-AGI-3 replaces static grid puzzles with turn-based games that ship with no instructions, no stated rules, and no stated goal: the agent observes a rendered frame, emits an arrow key or an (x, y) click, receives a new frame, and must infer the mechanics purely by acting. The paper's deliberately unfashionable thesis is that a training-free, algorithmic explorer beats LLM-based agents here — because an LLM agent is interaction-limited, gating every environment step on a model call and so managing only a few thousand interactions where the environment allows on the order of 96,000.

The system has two halves. A lightweight vision front-end segments each frame into single-color connected components, heuristically masks the status bar so UI chrome is not mistaken for game objects, and sorts the rest into five priority tiers by size, morphology, and color. A directed graph then records the environment: each visited state is a node keyed by a hash of the masked frame, and each tried action is an edge annotated with its outcome, priority tier, tested status, successor state, and shortest-path distance to the nearest unexplored frontier. A three-phase, frontier-driven policy picks a random untested action within the active priority threshold, else navigates to the nearest state that has one, else relaxes the threshold — a systematic, breadth-first-flavored coverage strategy that handles arrow (|A|=4), click (4,096), and combined (4,100) action spaces.

The payoff is a 3rd-place finish on the ARC-AGI-3 Preview Challenge private leaderboard, a median of about 30 of 52 levels solved in full 8-hour runs, and 12 private levels on the official submission (depressed by a candidly documented reset-marking bug that caused looping). Performance degrades on very large state spaces, and the agent explores but never models — recording transitions without inducing a rule that explains them.

Inductive program synthesis usually needs two things handed to it: a domain-specific language of primitives to write programs in, and a search strategy to find the right program fast. DreamCoder's wager is that both can be learned. It reframes domain expertise as acquiring a language for thinking about that domain — a set of reusable concepts — together with the skill to search efficiently within it. So instead of fixing a DSL up front, the system starts from a handful of primitives and grows its own library, while simultaneously training a neural network that learns how to steer the search through it.

The engine is a wake-sleep loop that alternates between solving problems and reorganizing knowledge. In the wake phase, a neural recognition model maps a task's examples to a distribution over which library components are likely useful, guiding an enumerative search until a program fits. In the abstraction ('sleep') phase, a compression step inspects solved programs and promotes frequently reused sub-computations into new named abstractions — inventing higher-level concepts like map or fold precisely because they shorten the solution set. In the dreaming phase, the recognition network is retrained on replays (real solved tasks) and fantasies (programs sampled from the current library and executed to synthesize examples), so the searcher improves between real problems. This is Bayesian program learning: the library is a generative prior, solving is posterior inference, and the recognizer is an amortized approximation that makes search tractable.

Demonstrated across eight-plus domains — list and text processing, symbolic regression, LOGO-turtle drawing, block towers — DreamCoder rediscovered foundational concepts it was never told: it reinvented core functional combinators from primitives and, from physical data, reconstructed vector algebra and laws like Newtonian gravitation and Coulomb's law. The grown libraries are human-readable, and the system bootstraps — later concepts build on earlier ones, so accuracy and search speed compound over rounds. That pairing of an interpretable, self-grown DSL with a learned search policy is the template for modern neuro-symbolic ARC solvers, and a direct model for this project's record → synthesize → replay-verify world-model loop.

ARC-style abstract reasoning and low-level sequential control look like different problems. This paper argues they are, from a model's perspective, the same capability: completing structured token sequences. It reframes what a pretrained LLM is — not a language model but a general sequence pattern machine — and asks how much abstract-reasoning competence a frozen, off-the-shelf model already carries with no fine-tuning at all.

Everything runs through in-context prompting on a frozen model. Structured data is serialized to token sequences: ARC grids become rows of number/character tokens laid out as ASCII art, with demonstration input to output pairs concatenated in the prompt and the model asked to continue the held-out output. The authors organize the capability into three modes — sequence transformation (map input tokens to output tokens, how they cast ARC), sequence completion (extrapolate procedurally generated PCFG and functional patterns), and sequence improvement (condition continuations on returns/rewards to extrapolate better trajectories, an in-context form of trial-and-error). For the pivotal token-invariance study, the alphabet is permuted to arbitrary vocabulary tokens so surface semantics are stripped while relational structure survives.

A frozen LLM solves a non-trivial subset of ARC tasks zero-shot with no visual encoder and no ARC-specific training, and its pattern completion partially survives random token remapping — the paper's central evidence that the skill is structural, not memorized. Pushed into robotics, the same machinery extrapolates motions from keyframes and, via least-to-most reward-conditioned prompting, discovers a closed-loop CartPole controller. The authors are candid about practical non-viability today — latency, context limits, compute — and present the work as a capability demonstration, not a deployable system. The through-line for ARC research: a frozen LLM already contains a surprisingly general, semantics-agnostic pattern engine that can be prompted, not trained, to attack abstract reasoning.

In December 2024 OpenAI's o3 reached about 87.5% on ARC-AGI-1, a jump widely reported as a step toward general intelligence. Pfister and Jud accept the number as genuine and impressive, then use the moment to ask what the benchmark actually measures. Their answer turns on a distinction: skills are competence on a bounded, familiar class of problems, while intelligence is the capacity to acquire new competence in unfamiliar situations. On that reading, a very high ARC score demonstrates the former without touching the latter.

The argument hinges on how o3 succeeds. The authors characterize its approach as massive trialling of combinations of predefined operations — a brute-force search over candidate grid transformations, filtered by evaluation at high compute cost. ARC-AGI-1's tasks are solvable by recombining a finite repertoire of operations, so this strategy is highly effective there. But real-world problems, they argue, generally lack a pre-enumerable operation set and lack cheap ways to test candidate solutions in advance, so a recipe built on exhaustive recombination does not extend to open-ended settings. From that gap they advance a positive definition of intelligence: achieving diverse goals in diverse worlds with minimal prior knowledge, centering breadth of transfer and economy of priors over peak performance on any one distribution.

The paper is conceptual, not empirical — no new models or experiments, just a framework and a critique. Under its lens, ARC-AGI-1 is one narrow slice: a single grid representation with a solvable-by-recombination structure, leaving the diversity-of-worlds axis essentially unmeasured even at 87.5%. The authors sketch requirements for a stronger benchmark — far greater task and world diversity, explicit accounting of allowed priors, and a design that forbids solve-by-recombination and pre-testable-solution shortcuts. Read as a corrective to the ARC-Prize narrative, it stays sympathetic to Chollet's skill-acquisition-efficiency framing but insists that clearing one hard benchmark is not the same as measuring general intelligence — a caution pointing toward the interactive, instruction-free setting of ARC-AGI-3.

The paper is a rebuttal. The prevailing story holds that ARC's resistance to LLMs proves program synthesis is the only viable path; the authors — from the lineage that pioneered test-time adaptation and won the 2023 ARCathon — argue the opposite. Their claim is that ARC problems are fundamentally perceptual and qualitative rather than symbolic, that the space of possible transformations is too large for hand-guided program search, and that deep learning is precisely the paradigm that has repeatedly learned hard abstractions in vision and language. So rather than abandon it, commit harder — including by training the network at test time on the very task being solved.

The methodological heart is treating both the network and the optimizer as part of inference. The base is a LongT5 encoder-decoder, chosen because its non-causal encoder attention lets every grid token attend to the whole input at once — grids are serialized row-major as numeric tokens. Pretraining leans on code (argued to be "more logical and less ambiguous"), multi-task data, and synthetic riddle generators, with an auxiliary head that predicts the underlying DSL function name. Test-Time Fine-Tuning repurposes each task's demonstration pairs as held-out riddles, multiplies them via color permutations, the D4 spatial group, and demo-order shuffling, then full-parameter fine-tunes before predicting. AIRV (Augment, Inference, Reverse-augment, Vote) applies invertible transforms to the input, infers on each view, inverts each prediction back to a common frame, and majority-votes.

Empirically the gains are large and stackable: from a ~5% zero-shot base, AIRV yields roughly +260%, and adding TTFT a further ~+300%, reaching 58% on the ARC private test set — the best score at the time — under tight compute (a single P100 within a couple of hours). Ablations note scale helped AIRV more than TTFT, and encoder-decoders beat causal decoders. The takeaway: the winning ingredient is learned, on-the-fly abstraction acquisition, not symbolic search.