Run comparison
Compare two scope-compatible runs side-by-side using canonical score deltas, evidence coverage, and per-finding lifecycle changes.
Key points
- Inputs are a current run ID and an explicit baseline; the response carries score deltas and finding diffs when the evidence is comparable.
- Findings classify as new (only in B), resolved (only in A), or regressed (still present + same severity).
- The API returns score and finding deltas only when both runs have compatible scope and decision-grade lineage; otherwise the comparison is inconclusive.
- Available from the UI at /runs/compare?a=A&b=B and from the API at GET /v1/runs/{current}/diff?baseline={baseline}.
01
Practical comparisons
- Pre-deploy vs. post-deploy of a prompt change — confirms a hardening landed and did not introduce new gaps.
- Last night vs. tonight — surfaces silent regression in upstream model behavior.
- Production vs. staging — quantifies the defense gap between environments.
02
What it does not do
- It is not a longitudinal trend view — compare two specific runs, not a window of runs.
- It does not merge or rerun — produce both inputs from real run executions before comparing.
- It compares within a tenant only; cross-tenant comparison is intentionally not exposed.