Run comparison
Compare two scope-compatible runs side-by-side using canonical score deltas, evidence coverage, and per-finding lifecycle changes.
Run comparison puts two scope-compatible runs side by side: canonical score deltas, evidence coverage, and the per-finding lifecycle between them. It answers the questions teams actually ask — did the hardening land, did something regress, how wide is the staging gap.
The comparison is deliberately narrow, and this page draws the line: two specific runs, not a trend window.
Key points
- Inputs are a current run ID and an explicit baseline; the response carries score deltas and finding diffs when the evidence is comparable.
- Findings classify as new (only in B), resolved (only in A), or regressed (still present + same severity).
- The API returns score and finding deltas only when both runs have compatible scope and decision-grade lineage; otherwise the comparison is inconclusive.
- Available from the UI at /runs/compare?a=A&b=B and from the API at GET /v1/runs/{current}/diff?baseline={baseline}.
Practical comparisons
- Pre-deploy vs. post-deploy of a prompt change — confirms a hardening landed and did not introduce new gaps.
- Last night vs. tonight — surfaces silent regression in upstream model behavior.
- Production vs. staging — quantifies the defense gap between environments.
The recurring comparisons: pre- and post-deploy of a prompt change, confirming a fix landed without new regressions; last night against tonight, surfacing silent upstream model drift; production against staging, quantifying the defense gap between environments.
What it does not do
- It is not a longitudinal trend view — compare two specific runs, not a window of runs.
- It does not merge or rerun — produce both inputs from real run executions before comparing.
- It compares within a tenant only; cross-tenant comparison is intentionally not exposed.
It does not merge or rerun anything — both inputs must come from real executions. It is not a longitudinal view, and it stays within a tenant: cross-tenant comparison is intentionally not exposed.
Related resources