Still testing this

100 percent on ARC-AGI-3

The scorecard is public: d4c56c67. All 25 environments, all 183 levels, score 100.0, and a working replay link under every game.

One Pi session played the whole thing across 22.4 hours. It spent $1,349 on the model while I contributed about four minutes of keyboard time, spread across seven short messages.

The agent had one tool.

22.4hone session
1,455fabric_exec of 1,457 calls
15compactions survived
4 minof human typing

The setup

The machine ran Pi with pi-fabric installed. Fabric swaps the usual box of small tools for a single one called fabric_exec, which takes a type-checked TypeScript program. Inside that program the model can call Pi's core tools, agents, memory, state, compaction, and whatever MCP servers sit on the box, and if the program's shape is wrong the type checker throws it back before anything runs.

The model was gpt-5.6-sol at maximum thinking. My prompt was a URL and one line:

https://docs.arcprize.org/

Help me achieve 100% across all 25 games in ARC AGI 3 locally.

I also loaded the fabric-schema skill, which hands the session a ledger. Every claim goes down in writing with falsifiable evidence attached, and the plan voids itself the moment that evidence goes stale.

What the model did

Hour one went to plumbing. The model pulled all 25 games down through the public Arcade wrapper, instantiated them locally, and parsed each game's source until it could simulate what the mechanics do.

The core of the run is state-space search. For every game the model wrote a simulator from the parsed source, hashed frames and game states so the search could tell where it had already been, and let breadth-first search grind with a timeout per level. When plain search stalled on a game it slowed down and specialized. One game got a constraint solver for rule rotations. Another needed collision-aware ordering, because it asks you to assemble loose parts onto a canvas without knocking things into each other. KA59 level 7 took recursive reasoning about how bombs re-arm and shove pieces around, and the mirror puzzle in AR25 came down to propagating constraints across reflections.

About forty minutes in I sent my shortest message of the session: "Stop the agents." The model had spawned parallel helpers to poke at games, and a single auditable line of work was worth more to me than the speed. The rest of the campaign ran as one sequential loop.

Wins had to be earned on paper. A game counted when its transcript showed WIN evidence, and the model kept a completion ledger the whole way through. Near the end it re-verified the full ledger from scratch by parsing bounded logs for ground truth, which is how we knew the 25/25 was real before touching the network.

ONE SESSION, 30.3 HOURS OF LOG1,457 Fabric call messages · 15 compactions · 7 human messages
0H6H12H18H24H30H01 kickoff02 which runs03 stop agents04 one card05 persist06 404 report07 new card25/25 VERIFIED LOCALLY · 22.4HSCORECARD REQUESTED
Assistant messages carrying Fabric calls, bucketed per 30 minutes from the raw session log. Diamonds mark compactions. Pins are the human messages, fanned into annotation lanes with kickoff pinned at zero. Every game already had a WIN marker by hour 13.2 (see the next chart); the quiet middle stretch is overnight search that emitted few calls per hour, and the ink-hot spike before the 22.4-hour gate is the fresh end-to-end audit. Hover the bars, diamonds, pins, and the two milestone rules for what was happening hour by hour.
FIRST WIN PER GAME, OVER TIME25 games banked in 13.2h · the last nine hours moved no dot
0H4H8H12H16H20H24H0510152025STUCK IN THE OPEN · 9.2H · NO NEW GAMESpurging false positives · shortening plans under score capsKA59 LEVEL 7 · BOMB FIGHTSCORECARD REQUESTED22.4H · FRESH AUDIT PASSES 25/25
A game enters the curve at the earliest log line that ties its key to a WIN marker, which is how batch verification sweeps show up as steps. The hatched plateau is honest: from the twenty-fifth game at hour 13.2 until the fresh audit at 22.4, the session added zero new games. Those hours went to re-running plans, cutting actions where score caps punished length, and the late fights named below the axis. Hover any dot for the game and its replayed effort; the plateau, gate, and battle markers carry their own notes. Data: the session jsonl.

The replay

ARC scoring weights each game and caps actions, so a bloated plan costs points even when it wins. Once every game had a verified win, the model squeezed its plans and then distilled the campaign into data. arc25/manifest.json holds the exact versioned game IDs plus all 6,164 public actions, and a small runner replays them against a single scorecard, offline first and then online with four workers streaming games into the same card.

THE REPLAY, DRAWN6,164 public actions · 183 levels · 25 games
AR258L · 253ABP359L · 365ACD826L · 70ACN046L · 177ADC226L · 432AFT096L · 75AG50T7L · 291AKA597L · 275ALF5210L · 669ALP858L · 79ALS207L · 313AM0R06L · 231AR11L6L · 110ARE868L · 522AS5I58L · 251ASB268L · 132ASC256L · 136ASK488L · 315ASP806L · 175ASU159L · 115ATN367L · 91ATR876L · 132ATU939L · 185AVC337L · 167AWA309L · 603A
Every recorded public action from arc25/manifest.json, drawn left to right at one fixed pitch so density stays comparable between rows. Tick height encodes directional actions 1 through 5; full-height ink strokes are pointer clicks on a coordinate (ACTION6). Rows where clicks merge into a solid band are games solved almost entirely by pointing. Row readouts give expected levels and the replayed action count. Hover a row to see the exact board the plan ran against, before and after.

Publishing took a second try. The first card looked healthy on the scoreboard page while every inner recording link returned 404. Recordings go live only after a card closes, and in that window something had gone wrong, so we opened a fresh card, re-ran the replay, and clicked through all 25 recording links in a live browser tab using the CDP skill. The card linked at the top is the one that survived that audit.

TWO PUBLISHED CARDSpublic replay files, probed after close
bd7c7519
score page 200scoreboard rendered
404404404404404404404404404404404404404404404404404404404404404404404404404
0 / 25 replay files public
d4c56c67
score page 200scoreboard rendered
200200200200200200200200200200200200200200200200200200200200200200200200200
25 / 25 replay files public
The first published card reported 100 while every inner recording link failed after close, so the evidence stayed private. The replacement card was probed link by link in a live browser tab before we trusted it. Cell order follows the manifest; one cell per environment. Hover any cell to see the board that environment plays on and the effort it took.

The campaign survives as a small git repo with tests. Anyone with the local environments can check the whole claim:

uv run python -m unittest discover -s tests
uv run python -m arc25.replay --mode offline

Why Fabric carried it

The session log shows 1,457 tool calls, and 1,455 of them are fabric_exec. Each one of those calls is a whole program: grep the repo, parse four logs, run a solver attempt, persist the winning plan, hand back twenty lines. The conversation sees the twenty lines while the sandbox keeps the megabytes.

That compression is what let a 19.9 megabyte session log stay coherent for 22 hours through fifteen compactions. After each compaction the schema ledger and the manifest sitting on disk gave the session its spine back, and when context thinned out the memory provider pulled the goal back into view.

The single flat tool schema did quieter work. At no point in 22 hours did the model fumble the interface, because it spent the whole run writing TypeScript against APIs it had already seen. Bad programs died in the type checker before they could spend a token.

The human part

My seven messages fit in one breath: an early clarification about which runs the model was reading, the stop order for the agents, an upload request, a persist-and-commit request, one bug report about the 404 recordings, and a yes for the new card.

The clarification is the one I'd point at. Early on, the model was quoting action counts from runs that other agents had published, mostly PRO-LONG's, alongside its own plans. I asked which source it meant, and from that message onward every claim in the session pointed at local evidence. Four minutes of typing, and most of it was steering.

Links

Have a lookmonotykamary/pi-fabric / main