Control

Controls. A policy is a list of rules. Before a tool call runs, the engine evaluates the rules against the call and the run so far, and the call is allowed, stopped, or held for a person.

The verdict above comes from the same policy engine your agents ask before each tool call, so what you see here is what your agent would get.

#How to: stop a tool call

This stops your agent calling a tool when a rule says no. The example stops refunds over a threshold on a disputed charge.

1. Write the policy

   In the app, open Controls and select New policy, or start from Templates → (Start from a template while there are none). Give it a Policy name, then write its Rules (select Load example to start from a working rule). Select Simulate against history → to see which of your past runs it would have stopped, then Save policy version.

   Expected output

   The policy appears under Saved policies. Simulation lists the calls it would have stopped, and calls no model.

2. Assign it to your agent

   On the same page, under Assignments, select Add assignment. Set Agent to the name your runs carry (your SDK's runName or run_name), choose the Policy, and set Mode to Block: stop the call, or hold it for approval. Select Save assignment. Choose Record only first to watch what it would stop without stopping anything.

3. Ask Runback before each tool call

   Turn on decide, then check each tool call before running it. If you use the Vercel AI SDK's dbg.tools(...), this happens for you.

   Copy

   // agent.mjs: npm install @runback/sdk
   import { startRun } from "@runback/sdk/core";
   
   // decide: true asks Runback before each tool call, against the policies
   // assigned to this agent name in Controls › Policies › Assignments.
   const run = startRun({ runName: "refund-agent", input: "Refund order 8842", decide: true });
   
   const args = { customer_id: 8842, amount: 250 };
   const d = await run.decide("issue_refund", args);
   if (d.allowed) {
     // ...call your real issue_refund tool here...
     console.log("allowed: issue_refund ran");
   } else {
     console.log(`${d.decision}: ${d.detail ?? d.rule}`);
   }
   
   await run.finish({ output: d.decision, status: "success" });

   Copy

   # agent.py: pip install runback-sdk
   from runback import Collector
   
   # decide=True asks Runback before each tool call, against the policies
   # assigned to this agent name in Controls › Policies › Assignments.
   run = Collector(run_name="refund-agent", input="Refund order 8842", decide=True)
   
   args = {"customer_id": 8842, "amount": 250}
   d = run.decide_tool_call("issue_refund", args)
   if d.allowed:
       # ...call your real issue_refund tool here...
       print("allowed: issue_refund ran")
   else:
       print(f"{d.decision}: {d.detail or d.rule}")
   
   run.finish(output=d.decision)

   Expected output

   With the policy assigned in Block mode, the refund prints deny: and the rule's reason, and your tool is never called. A call the policy allows prints allowed: issue_refund ran.

4. See what was stopped

   Open Controls. The stopped call is under What the rules stopped or held, in the latest calls a rule stopped or held, linked to its run, where the step shows as stopped.

#Troubleshooting

#Rules

A rule is one of two kinds:

Predicates combine with and, or and not:

A rule whose references do not resolve against the call is reported as not evaluated, never as passed.

#Where the decision happens

A rule with effect: "require_approval" holds the call for a person in Controls › Approvals instead of stopping it. From Starter

#Try a policy on past runs

In Controls, “Simulate against history” evaluates a draft policy over your last 100 recorded runs and shows which calls it would have stopped. No model is called. It needs a signed-in session. From Scale

#Alerts

Alerts fire on five triggers (run_failure, error_rate, cost_spike, silence, drift) and deliver to six channels: email, Slack, webhook, Microsoft Teams, PagerDuty (Events v2) and Opsgenie. Configure rules in Controls › Alerts. Sample runs (the demo data and the one-click sample run) never fire an alert and never count as an agent reporting. From Starter

#Webhooks

Every alert type sends the same body:

{
  "type": "silence",
  "subject": "refund-agent has been silent for 31m",
  "lines": [
    "No run from refund-agent since 2026-10-10T02:29:00.000Z (31 minutes; the rule allows 30).",
    "It normally runs at this time: it did in 4 of the last 4 weeks."
  ],
  "severity": "high",
  "dedupe_key": "critical_gap:refund-agent:stale",
  "link": "https://runback.dev/app/coverage",
  "sent_at": "2026-10-10T03:00:12.000Z"
}

Every delivery is signed. Each rule gets its own signing secret when it is created, shown once; New secret in Controls › Alerts issues another, and the old one stops verifying immediately. A rule created before signing existed sends deliveries without a signature until you issue one, and Controls › Alerts marks it “no signing secret”.

Verify against the raw request body, before parsing it, and reject a timestamp more than 5 minutes from your clock:

// verify.mjs: Node 18+. Returns true only for a fresh, untampered Runback alert delivery.
import { createHmac, timingSafeEqual } from "node:crypto";

/**
 * @param {string} secret     the rule's signing secret, shown once when it was issued
 * @param {string} rawBody    the request body exactly as received, before any JSON parsing
 * @param {string} timestamp  the Runback-Timestamp header (Unix seconds)
 * @param {string} signature  the Runback-Signature header ("v1=<hex>")
 * @param {number} [nowSeconds]
 */
export function verifyRunbackSignature(secret, rawBody, timestamp, signature, nowSeconds = Math.floor(Date.now() / 1000)) {
  const ts = Number(timestamp);
  if (!Number.isFinite(ts) || Math.abs(nowSeconds - ts) > 300) return false;
  const expected = "v1=" + createHmac("sha256", secret).update(timestamp + "." + rawBody).digest("hex");
  const a = Buffer.from(expected);
  const b = Buffer.from(signature || "");
  return a.length === b.length && timingSafeEqual(a, b);
}

PagerDuty and Opsgenie de-duplicate on dedupe_key. A delivery that fails with a timeout, 408, 429 or 5xx is retried with backoff, up to 5 attempts for silence and drift alerts and 3 for alerts raised as a run lands; any other 4xx is final. Recent deliveries in Controls › Alerts lists each delivery and its outcome.

#Next

A stopped call is a finding. Prove the fix turns it into a test.