What is the best way to handle secrets and API credentials an AI agent needs at runtime?

The best way to handle secrets and API credentials for an AI agent at runtime is to retrieve them just-in-time from an enterprise secrets manager using workload identity, ensuring that agents do not have standing access to sensitive credentials. This approach should be complemented by robust runtime enforcement mechanisms that mediate all agent actions and tool invocations.

Here are concrete controls for managing secrets and API credentials:

• Just-in-Time Credential Retrieval: Agents should retrieve secrets and API credentials just-in-time from an enterprise secrets manager using workload identity, rather than having them hardcoded or persistently stored. This aligns with the NIST AI RMF function of Govern, ensuring secure management of AI system components.
• Tool Broker for Authorization: All tool invocations should pass through a tool broker that validates each call against the agent's identity, an active intent token, and established policy. This broker enforces Intent-Based Access Control (IBAC), which normalizes heterogeneous tool calls to canonical actions and applies consistent policies. This addresses the OWASP LLM Top 10 risk of "LLM01: Prompt Injection" by ensuring that even if an agent is manipulated, its actions are still subject to strict authorization.
• LLM Gateway for Model Access: An LLM gateway (AI proxy or AI firewall) should be placed in front of every model invocation to enforce authentication, apply content policies, perform PII detection and redaction, rate-limit, and attach cost accounting. This acts as a chokepoint for uniform policy enforcement and helps prevent "LLM07: Insecure Output Handling" by inspecting and redacting sensitive information.
• Sandboxing for Code Execution: Any tool that executes generated code or processes untrusted data should operate within a properly isolated sandbox. This sandbox should have strict resource limits, no outbound network access except through the tool broker, ephemeral filesystems, and no access to the agent's credentials. This control helps mitigate "LLM04: Insecure Plugin Design" and "LLM05: Excessive Agency" by limiting the potential impact of compromised tools or agents.
• Runtime Inspection and Enforcement: Implement a system like DefenseClaw that performs runtime inspection of prompts, responses, and tool calls, enforcing policies based on severity (allow/warn/block). This provides a practical control plane that scans skills, MCP servers, and plugins before they run, and detects risky behavior such as hardcoded credentials or sensitive file access attempts. This addresses the OWASP LLM Top 10 risk of "LLM05: Excessive Agency" by ensuring that agent actions are continuously monitored and controlled.
• Unified Governance Layer: Utilize a unified governance layer, such as Intent-Based Access Control (IBAC), to normalize tool calls and enforce consistent security standards across diverse agent runtimes. This prevents policy fragmentation and secret leakage by ensuring that context-based policies can be uniformly enforced. This aligns with ISO/IEC 42001 control 8.2.1 "AI system security policy" by providing a consistent and auditable policy framework.