Attack scenarios
A scenario describes an adversary persona, the technique they exercise, and the assertion that decides whether the defense held.
A scenario encodes one adversary question: with this persona, this technique, and this assertion, would the defense hold? Scenarios keep tests comparable across runs because the assertion — not the model's mood — decides the outcome.
This page separates the scenario families that ship with the engine from the custom scenarios a tenant can author, and explains where each is appropriate.
Key points
- Saved custom templates record `updated_at`; each run preserves the concrete request it executed for later review.
- Built-in LLM probes retain stable category and technique attribution where the corpus defines it.
- Selected built-in LLM scenario families include versioned paraphrase variants for scoped robustness checks.
- Custom scenarios can be added via the scenario authoring API.
Built-in scenario families
- Prompt injection / jailbreak (LLM01).
- Sensitive information disclosure (LLM02).
- System-prompt leak (LLM07).
Built-in families map to OWASP LLM Top 10 identifiers so a finding can be read next to the taxonomy without translation. Families are versioned: when a probe corpus gains paraphrase variants, category and technique attribution stay stable across versions.
Custom scenarios
- Save a title, category, optional tags, and a schema-valid RunRequest template through the scenario API.
- Tenant and plan boundaries are enforced before the template is stored.
- The engine evaluates the saved template through the normal evidence pipeline; executable grader uploads are not supported.
Custom scenarios are templates, not scripts: a schema-valid run request plus metadata. The engine executes them through the same evidence pipeline as built-ins, which is why a custom scenario cannot bypass grading or claim a state the evidence does not support.
Related resources