Huawei's Memento 3 clears all 25 ARC-AGI-3 games with 56% fewer actions than humans

Memento 3 runs on Claude Opus 5 as its base model, with no weight updates and no fine-tuning for the games. Under ARC Prize's standard evaluation harness, Claude Opus 5 on its own averages about 40.7 RHAE, so the same underlying model gained 59.3 points when wrapped in the new agent framework.

The method is called Code as Model. While playing, the agent maintains two artifacts. The first is a Rulebook, a Markdown document recording what actions trigger which effects, how levels are completed and which rules remain unresolved. The second is an Executable World Model: Python code the agent writes to act as a simplified simulator of the game. It tries routes inside its own simulator before committing to a move.

When its understanding proves wrong, the agent revises. In one puzzle it initially assumed a special floor tile mirrored the pattern a character carried according to the direction of entry; testing showed the pattern rotated 90 degrees clockwise regardless of direction. The agent rewrote the rule and the matching code, then re-ran earlier moves and compared frame by frame against the real 64-by-64 game screen, continuing to correct the model until the simulation matched. The team used Git version control to track revisions; in one symbol-decoding game the rulebook went through 11 versions.

Controlled experiments across three games showed the rulebook's value: with it, the agent needed 617 actions; with only the code world model, 677, roughly 9% more actions and about 18% more agent turns. Running two world models at once, sharing the action history but testing different rule hypotheses, cut the actions needed to finish the game wa30 from 899 to 597, about 34% fewer. The team noted the dual-model gain has been verified only on wa30.

The researchers also ran Memento 3 on Atari Pong, where the ball moves and bounces in real time. The agent revised its bounce model as trajectories contradicted it, then the learned policy was handed over to code and Claude Opus 5 was no longer called. Across three different openings it won 21-0 each time. Without a world model, letting the model choose each move from the screen produced a score of -19, close to the -20.7 of random input. Memento 3 learned the control policy from roughly 9,500 frames of gameplay, while some conventional reinforcement-learning methods need hundreds of thousands to hundreds of millions of frames.

On the ARC-AGI-3 public set, Princeton's Continual Harness cleared 3 games with a score of 20.5, DreamTeam, developed with Nvidia researchers, cleared 6 with 38.1, and OPINE-World from independent researcher David Courtis scored 78.4, while Nvidia's NOOA reached 85.1. Independent developer Sergey Rodionov's baseline1 cleared all 25 games at 99. Memento 3 reached 100 and cut total actions to 7,518 from baseline1's 8,347. ARC Prize data show all 10 participants solved the amoeba puzzle r11l, while only 6 of 12 solved the symbol-decoding game tr87.

Costs and limits remain. The system ran Claude Opus 5 at extra-high reasoning effort, and exploration, code generation and verification consumed substantial tokens and compute. Experiments so far cover games with relatively clear rules; the team said whether the approach works in more complex and changeable real-world settings is still untested. The paper argues that recursive self-improvement need not alter model weights: an external world model that can be inspected, executed and rolled back lets a frozen general-purpose model iterate and improve in an unfamiliar domain.

Editor's Summary

Huawei's Noah's Ark Lab and collaborators report that Memento 3, an agent built on a frozen Claude Opus 5, completed all 25 ARC-AGI-3 public games and 183 levels with a perfect 100 RHAE score and about 56% fewer actions than human players. The system writes and revises its own rulebook and runnable world model, and it later won Atari Pong 21-0 in three games without further model calls. The team cautions that the work relies on heavy inference compute and has been tested mainly in games with clearly defined rules.