"Somewhere in the last dozen prompt revisions, this agent started approving refunds it should have escalated. Which one?" The naive answer is to re-run all twelve, one at a time, until you spot the flip. The correct answer is the same one git bisect gives you for a broken commit: binary search the timeline instead of walking it.

12 → 4

candidates in the timeline vs. probes needed to find the regression

O(log n)

re-executions instead of O(n) — the gap widens fast as history grows

The contract the search relies on

Binary search only works if the good→bad transition is monotone — the run is good at the start of the range and bad at the end, with no good candidate reappearing after the flip. That's the same assumption git bisect makes about a commit history, and it holds for the same reason: a regression is a state change, not a coin flip. Runback's bisect() takes that contract as given and does nothing you couldn't verify yourself — it just automates the probing.

Each probe halves the remaining range. Twelve candidates need at most four re-executions to name the exact one that flipped good to bad — not twelve.

bisect.ts

export async function bisect(
  count: number,
  isGood: (index: number) => boolean | Promise<boolean>,
): Promise<BisectOutcome> {
  let lo = 0, hi = count;
  let firstBad: number | null = null;
  while (lo < hi) {
    const mid = (lo + hi) >> 1;
    if (await isGood(mid)) lo = mid + 1;   // good -> culprit is to the right
    else { firstBad = mid; hi = mid; }     // bad  -> culprit is here or left
  }
  // firstBad is the regression's exact index, found in ~log2(count) probes
}

Every probe is a real re-execution — the captured run replayed under that candidate's model or prompt version, with the hybrid replay engine serving every recorded value except the one axis under test. That's what keeps each probe cheap: you're not re-running the whole agent from scratch twelve times, you're changing one variable and replaying the rest.

The probe sequence is returned alongside the verdict, not thrown away — "trust the binary search" is exactly the kind of claim this product exists to argue against, so you can see which four candidates were actually checked and confirm the logic yourself.

Why this matters more as the timeline grows

At 12 candidates, linear search costs you 12 re-executions in the worst case and bisection costs 4 — a meaningful but not dramatic difference. At 1,000 prompt revisions across a year of iteration, linear search costs 1,000 and bisection costs 10. The gap is logarithmic, so it's exactly the regressions that have been hiding the longest — the ones nobody wants to re-run a thousand times to find — where this matters most.

Bisecting a model regression in 4 probes instead of 12

The contract the search relies on

Why this matters more as the timeline grows