PROJECTS / 04
DEMONSTRATED
Deterministic failure-and-recovery testing for AI agent workloads.
Agent Rescue is the lab’s AI reliability and recovery work: reproducing agent failures on purpose and demonstrating the full recovery path — deterministically, with every step on the record. Built around the WebMCP Challenge.
Conceptual artwork — a light path rerouting around a failed node.
01 · THE DEMONSTRATED SCENARIO
A job failed at 73%. Then it didn’t.
The demonstrated run: an AI job fails at 73% completion due to an HTTP 429 condition. The failure is identified, an alternate provider is selected with human approval, the job resumes from preserved progress — and recovery completes.
STEP 01
The job fails at 73%
HTTP 429
An AI workload running through its task lifecycle halts at 73% completion, blocked by an HTTP 429 rate-limit response from its provider.
STEP 02
Failure identified
DIAGNOSIS
The failure condition is identified and recorded — a recoverable quota event, not an unrecoverable error.
STEP 03
Alternate provider selected
HUMAN APPROVAL
An alternate provider is selected to continue the job, and the switch is approved by a human operator. The authority boundary stays explicit — no silent failover.
STEP 04
Resumed from preserved progress
CHECKPOINT
The completed 73% is preserved. The job resumes from the checkpoint instead of restarting from zero.
STEP 05
Recovery completed
100%
The workload completes. The full failure-to-recovery loop is demonstrated end to end — deterministic and reproducible.
02 · WHAT THE RUN DEMONSTRATES
Four things most agent demos skip.
01
Failure detection
Recognizing a quota condition for what it is — and recording it — instead of retrying blindly or failing silently.
02
Human-in-the-loop switching
Provider selection that requires human approval, keeping the authority boundary explicit.
03
Progress preservation
Checkpointed state so interrupted work resumes instead of restarting.
04
Resumption to completion
A recovery that finishes the job — not a report about the job.
03 · WHY IT MATTERS
Reliability is the feature.
Real deployments hit rate limits, provider outages, degraded modes, and quota ceilings. An agent that can’t survive a 429 isn’t a system — it’s a demonstration. Deterministic failure testing turns reliability from a claim into something you can watch happen, on demand, with the evidence recorded.
EVIDENCE NOTE
This page documents the demonstrated scenario. Detailed run records and results are released through Lab Reports as they are cleared for publication.