The Durability Benchmark
Measuring durable capability in Frontier AI.
Frontier models can reason, use tools, and complete bounded tasks. Long-running systems still lose task state, drift from constraints, fail to recover, and require a person to rebuild the trajectory. Durable Intelligence Labs is convening a founding consortium to measure the complete deployed system under those conditions, including whether its alignment survives.
Measuring the gap between intelligence and autonomy.
CAIS reports rapid gains in cognitive capability, large variation across tasks, low full-project automation, and uneven risk performance. Together, these evaluations do not isolate whether the complete system remains capable and aligned through one long-running trajectory.
Current evaluations measure model intelligence, task capability, end-to-end automation, and safety. None isolates persistence across a continuous trajectory.Source: CAIS AI Dashboard, standard tables, accessed 2 September 2026. Values come from separate evaluations and are not directly comparable. Remote Labor Index projects represent more than 6,000 hours of professional work.
The protocol records useful capability and alignment retained after each event, every human intervention, the recovery path, and the final valid outcome.
Testing the complete system through state change.
The program is designed as a Humanity’s Last Exam-grade benchmark for durability. The founding consortium will define public practice environments and held-out official evaluations, with each run recording system configuration, state changes, interventions, recovery behavior, and outcome.
The benchmark defines durability as useful capability and governing alignment retained through time, failure, and state change.
- 01Ultra-long-horizon performance
Track progress, errors, and valid outcomes as task duration and dependency depth increase.
- 02Human-out-of-the-loop capability
Separate authorization from corrective labor. Record restarts, state reconstruction, reasoning repair, and hidden rescue.
- 03Endurance under state change
Perturb context, tools, evidence, environment, and model allocation. Score state retention and recovery.
The benchmark evaluates the complete system that produces the outcome. A bare frontier model provides the intelligence baseline. The harness creates the agent by adding memory, orchestration, tools, verification, guardrails, and recovery. Supporting infrastructure is disclosed and measured through its effect on the composite system. Each submission declares the full stack, exposes the trajectory, and produces a result that an independent evaluator can reproduce.
Defining and validating a public benchmark.
Durable Intelligence Labs is assembling researchers, frontier labs, and agent-stack builders to define the system under test, evaluation environments, intervention rules, scoring, disclosure, and independent reproduction.
Protocol work
- System-under-test and baseline classes
- Persistent environments and perturbation regimes
- Intervention, scoring, and disclosure rules
- Held-out evaluation and reproduction
Public release
- Durability protocol and reference implementation
- Practice and held-out evaluation suites
- Frontier-model and agent-stack baselines
- Reproduced results and a failure corpus
MIT CSAIL
Google DeepMind
ImperialWith model and harness builders contributing systems under test.
Engineering durable systems of intelligence.
The benchmark identifies where capability and alignment fail. Durable Intelligence uses those failure modes to develop the neutral durability layer around inference, test systems-level intelligence, and design non-bypassable controls for recursive self-improvement.
Measure capability and alignment before, during, and after state change.
Run heterogeneous models as one persistent system and test the composite result.
Keep recursive improvement observable, interruptible, and subject to external control.
If causal state survives model swaps, compaction, and failure, capability accumulates across a system instead of resetting every call. The benchmark is the measurement. Nenya and The Valley are the experimental systems.
Contribute to the protocol