Runback
Blog

How it works, in detail.

Engineering notes on tracing, replay and proof, with the code behind them.

ReplaySeptember 1, 2026

Bisecting a model regression in 4 probes instead of 12

git bisect for agent behaviour: binary-searching a candidate timeline for the exact model or prompt change that introduced a regression, in O(log n) re-executions.

Read it →
prompt v1
prompt v2
prompt v3
prompt v4
prompt v5
prompt v6
prompt v7
prompt v8
gpt-4o
gpt-4o-0513
gpt-4.1
claude-sonnet-4-6
Scripted candidates, the real bisect algorithm. 12 candidates, one regression, unknown which.

Earlier posts

Replay 3 posts

  1. July 14, 2026We held a real data race to one fixed scheduleA stress test of Runback's scheduler: a genuine, lock-free data race gives the same result on every run under one fixed schedule — tested in public CI, including the load-dependent flake that CI found.
  2. July 14, 2026Why replay-from-step-N is harder than it looksWhy capturing the request at the wrapLanguageModel boundary — not at your own call sites — is what makes replay mean something.
  3. July 14, 2026Debugging an AI agent with time-travel replayA walkthrough of a real failing agent — from error-first navigation to editing the exact captured request and replaying it.

Proof 2 posts

  1. July 14, 2026What a hash-chained audit trail actually provesThe exact algorithm behind Runback's signed, append-only audit record — and what it means for EU AI Act and APRA CPS 230 compliance.
  2. July 14, 2026Redacting secrets before they ever leave your processHow @runback/redact scrubs API keys, JWTs, SSNs, and card numbers in-process, before anything is sent anywhere.

Govern 2 posts

  1. September 1, 2026How we auto-mine production incidents into regression testsA failed run becomes a permanent golden-suite entry, deduped by signature so a thousand identical incidents collapse into one test that keeps checking itself.
  2. September 1, 2026Simulate a policy against history before you enforce it liveHow Runback's policy simulator checks a candidate rule against 100 recorded runs — no model calls, no replay — and names the exact runs it would have blocked.

Wire it in 1 post

  1. July 14, 2026Three ways to wire an agent into RunbackHow to pick between the Vercel AI SDK integration, an OpenTelemetry exporter, and the framework-agnostic manual recorder — with the actual commands for each.

New posts by RSS →