How do I secure a RAG (retrieval-augmented generation) system?

Securing a RAG system primarily involves safeguarding the integrity and confidentiality of its data and ensuring that the system's actions are aligned with its intended purpose, especially given the susceptibility of agentic LLM systems to logic-layer attacks.

• Implement robust RAG corpus governance to prevent poisoning and ensure data integrity. This includes source vetting, content classification on ingestion, change tracking with audit, periodic adversarial retrieval testing, and isolation between user-contributed and curated content. This addresses the OWASP LLM08 Vector and Embedding Weaknesses risk.
• Apply zero trust principles to context handling by tagging each context segment with provenance and trust levels. The model should be conditioned to respect these tags, ensuring that instructions from low-trust segments are treated as data, not directives. This helps mitigate indirect prompt injection (OWASP LLM01) by preventing untrusted content from being interpreted as instructions without explicit authorization.
• Treat vector databases as primary data stores for governance purposes due to their potential to contain reconstruction-grade representations of sensitive data. Implement access-controlled retrieval and per-tenant/source partitioning to prevent cross-context leakage and unauthorized access.
• Validate all model outputs and tool calls to prevent misuse and unsafe actions. This includes schema validation on every tool call, allowlisting tools/actions, and parameter constraints. For high-risk operations, implement action confirmation requiring human approval or a second model invocation with adversarial framing. This addresses OWASP LLM05 Improper Output Handling and OWASP LLM06 Excessive Agency.
• Enforce least privilege at the data layer so that the agent retrieves only the data needed for the task, and data classification flows with the data, ensuring downstream operations inherit restrictions. This prevents sensitive data from being summarized into unclassified outputs.
• Continuously evaluate the system's security posture through automated red-teaming and integration into CI/CD pipelines. This involves maintaining a golden dataset of known prompt injection variants, jailbreak attempts, and edge cases, and running automated red-teaming tools against every release candidate. This aligns with the NIST AI RMF function of Govern and Map by ensuring ongoing security assessments.