Attack simulations
A simulation is one controlled run of probes against your LLM application — scoped to an allowed target, executed by our engine, and graded against OWASP LLM Top 10 and MITRE ATT&CK.
Key points
- Every simulation has an explicit target_scope; out-of-scope targets are rejected.
- Run creation is idempotent: reusing the same Idempotency-Key returns the existing run, while model responses may still vary across new executions.
- A completed run persists findings and evidence artifacts. Its numeric score is decision-grade only when canonical persisted evidence supports a conclusive assessment; otherwise public and comparison scores remain N/A.
- Simulation profiles range from quick (≈5 minutes) to deep (≈30 minutes).
01
What a simulation produces
- Per-category findings tagged with OWASP LLM IDs and ATT&CK techniques.
- Per-category states (open / remediated / tested / untested) derived from conclusive persisted evidence.
- Eligible Starter+ runs can produce a checksum-verified evidence pack. LLM runs with eligible findings can also include request/response transcripts and a remediation playbook.
02
When to run one
- Before each release, against a staging endpoint.
- Weekly against production (smoke profile).
- After any change to system prompts, tools, or model versions.