PR Diff Red-Team Gate
The full workflow that runs when a pull request touches a watched prompt artifact. The GitHub App receives a pull_request webhook, parses the repo's .doesitdefend.yml, derives the OWASP categories the diff actually touches, runs a diff-scoped scan, compares against the most recent baseline on the same target (within the 30-day TTL), and posts back a check run + idempotent PR comment. The CI Integration concept is about the tooling (CLI + Action); this concept is about the workflow.
The PR Diff Red-Team Gate is the full workflow behind the check your team sees on a pull request: webhook in, diff parsed, categories derived, scan executed, baseline compared, verdict posted.
It is distinct from the CLI and Action tooling concept: that page describes what you install; this page describes what happens when the workflow runs.
Key points
- Diff-scoped: only the OWASP categories whose probes overlap with the changed files run. A README-only PR triggers zero scan minutes.
- Baseline-aware: regressions are computed against the previous run on the same target. A new target gets `no_baseline` (informational, never blocks).
- Idempotent: a PR amend / force-push rewrites the existing check run + comment instead of stacking.
- Schema-default: `llm_config.scenarios` falls back to the 10 OWASP categories when omitted, so the manifest stays lean.
Verdict modes
- regression (default) — fails the build only when the posture score drops past a material threshold (configurable; ±5 points by default).
- findings — fails the build whenever any new finding appears, regardless of baseline comparison.
- never — advisory mode; check always succeeds, comment carries the verdict for review.
Verdict modes exist because teams differ on how a gate should fail. Advisory mode always leaves the check green and carries the verdict in the comment; enforcing modes block. Either way the comment, not the check icon, is where the reasoning lives.
What the PR sees
- GitHub Check Run on the head commit with a one-line title and a markdown summary.
- PR comment with the diff-selected OWASP categories, per-finding evidence excerpts, and, for an eligible Starter+ run, a link to the full evidence pack.
- If the gate hits an error (invalid manifest, target unreachable, scan timeout), the check run sits at `neutral` so the build is never wedged.
When the gate cannot run — an invalid manifest, an unreachable target, a timeout — the check run states the error instead of inventing a pass. An unreadable gate is a broken control; a failed gate with a reason is a working one.
Related resources