A comparison ends in one of five verdicts, and the difference between them is where teams get burned: no_baseline looks like comfort, and inconclusive looks like noise. This page teaches you to read both correctly.
The baseline is automatic — the most recent valid run for the same target, within 30 days by default — so the verdict's meaning depends on knowing what that baseline actually was.
The five verdicts
- `regressed` — a new open finding appeared or overall risk increased beyond tolerance. Investigate before merging.
- `improved` — at least one prior finding was resolved, no regression appeared, and the scores did not worsen materially.
- `stable` — the finding set stayed the same and scores remained within tolerance. This describes the comparison; it is not proof of absolute security.
regressed blocks the merge; improved records resolved findings with no new ones; stable describes an unchanged comparison and is not proof of security. no_baseline is informational — never read it as an improvement — and inconclusive means the gate cannot trust its own evidence, so treat it as failed.
Score deltas vs. finding deltas
- Finding deltas (new / resolved / persistent) drive the verdict's binary fail/pass logic.
- Score deltas (overall_risk, detection, exposure, readiness) carry direction (better / worse / neutral) and magnitude — useful for trend reporting even when no findings changed.
- A scenarios array trim (e.g. diff-selected just LLM01) marks dropped findings as 'carried forward' instead of 'resolved' — so an old finding that wasn't re-tested doesn't fake an improvement.
Findings drive the pass/fail logic; scores carry the nuance. Watch for scope trims: a scenarios array that shrinks marks dropped findings as carried forward, not resolved — a smaller test is not a better result.