How does the NIST AI RMF address AI incident response?

The NIST AI RMF addresses AI incident response by requiring post-deployment monitoring and an AI/agent incident-response plan that includes detection, escalation, containment, communication, and learning. This falls under the MANAGE function of the NIST AI RMF, which focuses on prioritizing, responding to, and monitoring risks over time.

Specific controls and considerations for AI incident response include:

• NIST-MANAGE-4.1 mandates the presence of an AI/agent incident-response plan covering detection, escalation, containment, communication, and learning.
• NIST-MANAGE-1.3 requires that responses to the highest-priority AI risks are planned, tracked, and resourced.
• NIST-MANAGE-2.3 specifies that procedures must exist to deactivate, roll back, or safely retire AI systems that exceed risk tolerances, effectively acting as a kill-switch or rollback mechanism for agents.
• NIST-MEASURE-3.1 emphasizes the need for approaches to track identified and emergent risks, including monitoring, logging, and drift detection. This cross-maps to ISO/IEC 42001 monitoring.
• The Generative-AI Profile highlights risks relevant to LLM/agent stacks that could lead to incidents, such as confabulation/hallucination, information security issues (e.g., prompt injection, data exfiltration, insecure tool use), data privacy leakage, dangerous/harmful content, and value-chain/component integration risks. These risks should be considered in incident response planning.