Durable IntelligenceLabs
MenuClose menu

← Research

Scaling Discovery Does Not Require Forgetting: Verified Shared Archives Break the Multi-Agent Plateau on Anthropic’s Performance Take-Home

Lindell S. Cumes Durable Intelligence Labs

Read the paper Code and dataCite

PDF · 21 pages · 410 KB

The shared verified archive keeps descending after parallel and communicating search plateau.
Figure 1. The shared verified archive keeps descending after parallel and communicating search plateau. Anthropic’s performance take-home, four Claude Opus 5.5 agents per arm at medium effort, three seeds per arm launched together; all arms matched on machine, hours, effort, starting kernel, and protected evaluator. (a) Best verified cycles so far (lower is better); lines are seed medians and bands span the three seeds. Every arm starts from the 147,734-cycle kernel, so the curves enter from above the axis. The dashed line is CORAL’s published 1,103 cycles with Claude Opus 4.6, a different model. “False floor” marks the score that several CORAL-faithful agents concluded was near the floor. (b) The archive’s lead at the least favorable pairing, the best seed of the other arm minus the worst archive seed, every 15 minutes; it is positive at every checkpoint from 2.75 hours. Triangles mark two early values below the axis (−88 and −85).

Abstract

Tan et al. report that longer, parallel, and communicating searches “all … eventually plateau as the agents accumulate history” and that “the key to scaling discovery further is to forget”: discard every agent’s session and the candidate history, and restart from the best candidate. They never test resetting sessions while keeping candidates; we introduce a shared verified archive that does. It bounds conversations and keeps every evaluator-verified attempt with its source and cited tools, so different agents’ improvements compose. On Anthropic’s performance take-home, where their forgetting team reaches 1,077 cycles and CORAL 1,103 with other models, four Claude Opus 5.5 agents on the archive reach a median of 914 [913, 916] cycles over 3 seeds in 8 hours, against 948 for a CORAL-faithful team and 946 for the best of four isolated agents on matched machines. Both baselines’ medians stall at 946–956 cycles after 3 hours, where several CORAL-faithful agents concluded about 951 was near the floor and 7 of 12 fell into a no-op resume loop, which we name the false floor; from 2.75 hours the archive’s worst seed stays ahead of both baselines’ best seeds. On open problems, a team built 60 of its first 98 new bests on another agent’s attempt or tool and, in one run per arm, led every arm at 8 hours on Erdős’s minimum overlap problem, the first autocorrelation inequality, and AHC026.

Code and data

Per-seed anytime curves

  • evidence/analyst_budget_curve.csvTake-home best-so-far cycles per attempt for every seed of the main comparison, the original run, the added seed and the c2d-highcpu-56 rerun70 KB
  • figures/hero_data.csvFigure 1 data: the nine main-comparison take-home runs and the open-problem arms, every best-so-far step66 KB
  • evidence/curves_matched.csvOpen-problem best-so-far for every arm and problem on a 0.25-hour grid of matched working hours435 KB
  • kx/results/curves-completed.csvTake-home best-so-far curves for the model-change, mixed-family, seeded, open-weight and single-agent runs, with the agent and model of each new best18 KB
  • kx/results/analysis-completed.txtBest-so-far cycles at 15 to 480 minutes for each take-home run in curves-completed.csv, including the mixed-family, seeded, open-weight and model-change runs2.2 KB

The take-home task prompt

C₁ and Erdős constructions with their exact verifiers

  • anc/ac1-SX-A43.txt.xzon the new-constructions pageC1 construction at 3.97 working hours, 262,144 heights, bound 1.50274043592276697641…1.3 MB
  • anc/erdos-SX-A51.txton the new-constructions pageErdős minimum-overlap construction A51, 3,584 cells, bound 0.38085673888331742055…56 KB
  • anc/verifier/audits_crossfamily/exact.pyon the new-constructions pageIndependent exact checker for C1 and Erdős (GMP carry-free packed convolution, exact fractions)8.2 KB
  • anc/verifier/verify_all.pyon the new-constructions pageRuns every checker on the construction files and compares with the papers' values6.1 KB
  • anc/SHA256SUMSon the new-constructions pagesha256 of every construction and checker file40 KB
  • anc/README.txton the new-constructions pageFile formats, exact values and how to run the checkers6.1 KB

Cite

@misc{cumes2026scaling,
  title         = {{Scaling Discovery Does Not Require Forgetting: Verified Shared Archives Break the Multi-Agent Plateau on Anthropic's Performance Take-Home}},
  author        = {Cumes, Lindell S.},
  year          = {2026},
  url           = {https://durableintel.com/paper-societies-horizon}
}
Continue reading →