How do I extend a SOC 2 program to cover AI and LLM systems?

To extend a SOC 2 program for AI and LLM systems, organizations must implement controls for continuous discovery, robust identity and access management, comprehensive logging and monitoring, and thorough evaluation of AI systems. These controls help manage the unique risks posed by autonomous AI agents and LLMs, ensuring compliance and security.

• Maintain an AI System Inventory: Implement continuous discovery mechanisms to identify all AI agents, LLM runtimes, and agentic frameworks across endpoints, containers, and network traffic. This addresses NIST-MAP-1.5 AI system inventory by building a complete map of agentic tooling and correlating network observations with process inventories.
• Implement Strong Identity and Access Management: Integrate agent platforms with enterprise Identity Providers (IdP) for authentication and use workload identity for just-in-time retrieval of secrets from enterprise secrets managers. This helps mitigate risks like OWASP LLM06 Excessive Agency by ensuring agents operate with appropriate authorization and capability attenuation.
• Establish Comprehensive Logging and Monitoring: Ensure all agent actions, API calls, data movements, and identity events are logged to an enterprise SIEM, creating a forensically complete audit trail. This supports NIST-MEASURE-2.8 Transparency & accountability mechanisms and allows for the reconstruction of agent activities and blast radius calculation in case of an incident.
• Conduct Continuous Evaluation and Threat Modeling: Implement shadow evaluation, canary deployments, and adversarial dataset generation to validate changes and detect regressions or drift in AI systems. This addresses NIST-MEASURE-2.7 Security & resilience evaluated and helps mitigate threats like evaluation bypass and golden dataset poisoning.
• Manage Third-Party Risks: Establish controls for suppliers and third parties in the AI value chain, including model providers, data providers, and tool/plugin vendors. This aligns with NIST-GOVERN-6.1 Third-party / supply-chain risk policy and addresses OWASP LLM05 Supply Chain Vulnerabilities by scrutinizing third-party MCP servers and ensuring components conform to existing security contracts.
• Ensure Human Oversight and Data Governance: Design systems with explicit autonomy levels, human intervention points, and mechanisms for data classification propagation. This supports ISO/IEC 42001 A.9 Use of AI systems and ISO/IEC 42001 A.7 Data for AI systems, ensuring that derived data inherits the classification of its inputs and that humans can oversee and override agent actions.