How do I prevent XSS, SSRF, and command injection caused by passing LLM output into downstream systems?

To prevent XSS, SSRF, and command injection from LLM output, treat all LLM output as untrusted and implement robust validation and sanitization before it is rendered, executed, or passed to downstream systems. This directly addresses the OWASP LLM05 Improper Output Handling risk.

• Treat LLM output as untrusted and implement encoding/sanitization before rendering it in any user interface. This prevents XSS by neutralizing malicious scripts.
• For interactions with databases, use parameterized queries instead of directly embedding LLM output into SQL statements to prevent SQL injection, which can lead to command injection or data exfiltration.
• When LLM output is used as arguments for tools or functions, schema-validate tool arguments to ensure they conform to expected types and formats. This can prevent command injection and SSRF by restricting the values passed to external systems.
• Never use eval on model text, as this can directly lead to remote code execution (RCE) if the LLM generates malicious code.
• Implement output filtering and content classification on outgoing data to prevent the LLM from including sensitive context content in tool calls or external responses, which could be an exfiltration channel.
• For tools that execute generated code or process untrusted data, ensure they operate within a properly isolated sandbox with strict resource limits, no host filesystem access, and outbound network access only through a broker. This helps contain potential command injection attempts.