Behavioural drift
Catch a model quietly changing upstream, with no code change on your side.
Product
Everything Runback does, in the order you would use it.
Four stages, and the workspace around them — the same grouping the product uses internally, so what you read here is what you land in.
01 · Observe
Every decision, captured at the boundary.
Every model call — context, tools, tokens — recorded at the grain a decision is actually made. PII is redacted in-process, before anything leaves your app.
Runs · Alerts · Incidents · Coverage · Topology — what each one does →
02 · Replay
Re-run the real decision, not an approximation of it.
Re-execute from the exact captured context — tools, retrieval and the messages the model saw all held fixed. A different output means behaviour changed, and you know which change caused it.
Prompts · Playground · Datasets · Evals · Golden corpus · Compare · Corpus signals — what each one does →
03 · Gate
Block the violating action before it runs.
Write a rule and simulate it against ninety days of real decisions before it is live — so you know exactly what it would have stopped, and what it would have cost you. Then enforce it, and the block is sealed into the record.
Policies · Policy library · Causal attribution · Models · Model diff · Upgrade gate · Approvals — what each one does →
04 · Audit
Produce the record on demand. Let anyone check it.
A SHA-256 hash-chained, Ed25519-signed record of every decision. Tamper-evident, and verifiable by a regulator or an auditor without an account and without installing Runback.
Audit ledger · Compliance artifacts · Regulatory mapping · Administrative audit log · Open verification — what each one does →
The difference
Five things the observability tools do not do.
Bisect
Binary-search a candidate list for the exact change that broke it — O(log n) probes, each one recorded.
Judge calibration
Correct the LLM judge; every correction is fed back into future judging.
Fleet benchmarks
Your agents against anonymised peers in your vertical.
Cost attribution
Where the spend goes, by model and agent, with per-team budget caps.
Live demo · sample data, not customer figures
Want the mechanism rather than the surface? See how it actually works →