Advanced•10–16 hours

AI Security Incident Containment Drill

Exercise an AI-specific incident where prompt injection or a compromised tool has already affected a production workflow. Build evidence-preserving containment, reconciliation, scoped rollback, and regression closure.

incident responsecontainmentforensicsreconciliationrollbacksecurity regression

Scenario

Task

After an indirect prompt injection, a production agent called a write tool and a timeout left the final state unknown. Do more than disable the model: preserve evidence, bound the blast radius, verify the authoritative system of record, revoke the compromised capability, and restore service safely.

Step-by-step execution

1. Classify the incident and freeze evidence

Outcome: The team knows what happened and which runtime envelope was active.

Tasks

→Record model, prompt, retrieval, and tool versions
→Preserve traces without unnecessary secrets
→Identify user, session, and tool identities
→Separate confirmed effects from suspected effects

Checks

✓The audit trail is not rewritten retrospectively
✓Sensitive evidence has access control

2. Contain only the necessary blast radius

Outcome: The dangerous capability is stopped without an unnecessary full-system shutdown.

Tasks

→Disable the affected write tool or scope
→Revoke the compromised credential
→Move the workflow into degraded or read-only mode
→Block the known malicious source

Checks

✓Containment does not depend on model behavior
✓The critical action path is no longer available

3. Reconcile before retry

Outcome: Unknown side effects are reconciled against the authoritative source before any repeat execution.

Tasks

→Check the system of record
→Match idempotency keys
→Detect partial or duplicate writes
→Mark pending manual repairs

Checks

✓There is no blind retry after a timeout
✓Every uncertain action has a confirmed final state or an explicit manual owner

4. Restore known-good and close the regression

Outcome: Service returns in a controlled manner and the incident cause cannot silently recur.

Tasks

→Roll back the affected behavior envelope
→Run the security replay
→Perform canary and failback
→Minimize the exploit
→Add a permanent regression case

Checks

✓Recovery is verified with postconditions
✓The regression blocks recurrence of the critical attack path

Acceptance criteria

The incident timeline includes a runtime fingerprint
Containment is scoped and independent of model compliance
Unknown side-effect reconciliation is complete before retry
Known-good recovery is confirmed by authoritative postconditions
The production exploit becomes a permanent regression test