Runback
← Blog

Redacting secrets before they ever leave your process

Runback Team··security, engineering

Turn on debugging capture for an AI agent and you're capturing everything it saw and said — including whatever secrets happened to be sitting in that context. A customer's SSN pasted into a support ticket. An API key sitting in an environment dump the agent read. A JWT it fetched from an internal endpoint. Now that's sitting in your observability tool, on a screen your engineers share during a debugging session, unless it never made it there in the first place.

14 built-in detectors
emails, 8 provider API key formats, JWTs, bearer tokens, private keys, SSNs, card numbers — zero config
Raw captured event before and after in-process redactioncaptured, unredacted{ role: "tool",api_key: "sk-ant-a1b2...",ssn: "123-45-6789",model_id: "gpt-4o",tokens: 842}in-processbefore sendsent to Runback{ role: "tool",api_key: "[redacted]",ssn: "[redacted]",model_id: "gpt-4o",tokens: 842}token counts, latency, model id, and span shape survive — only secrets are touched
Illustrative example — the same detector logic runs on real captured events. Structural fields (model id, token counts) pass through untouched so monitoring keeps working.

In-process, not in-transit

@runback/redact is a separate package from the core SDK, and it runs inside your process, before anything is sent anywhere — not as a server-side scrubbing pass after ingest, which would mean the raw secret already left your infrastructure. Enable it with one option:

agent.ts
withDebugger(model, {
  runName: "my-agent",
  redact: {
    preset: "standard",
    redactKeys: ["x_internal_id"],        // also blank these object keys
    customPatterns: [{ name: "emp", regex: /EMP-\d{5}/g }],
    allowKeys: ["model_id"],              // never touch these
  },
});

What it catches

The "standard" preset is high-confidence detectors only, tuned to avoid false positives on real prose: emails, OpenAI/Anthropic/Groq/Stripe/GitHub/Slack/ Google/AWS keys, JWTs, bearer tokens, private-key blocks, SSNs, and Luhn-valid credit card numbers. "strict"adds phone numbers and IP addresses — noisier, so it's opt-in rather than default.

You can extend it: redactKeys blanks specific object keys outright regardless of pattern match, customPatterns adds your own regex-based detectors for anything domain-specific (an internal employee ID format, say), and allowKeys marks fields that should never be touched even if a pattern would otherwise match.

Redaction removes what a human or model shouldn't see, not the shape of the data an observability tool needs to be useful.

So what?

Token counts, latencies, model ids, and span structure are preserved even after redaction — so monitoring, cost attribution, and eval scoring keep working on a redacted trace. You don't have to choose between a debuggable agent and one that doesn't leak. The instrumentation code is three lines. The redaction is one option on top of it. Neither requires a third-party service in the loop, and neither requires you to trust that secrets got scrubbed somewhere downstream — they never left your process unredacted.