ChemCrow — agentic threat model

ChemCrow relies on LLMs optimized for chemistry. The primary threat is prompt injection or jailbreaking to bypass safety alignments, potentially allowing users to generate instructions for synthesizing restricted, hazardous, or dual-use chemical substances.

Not certain from the listing — however, the agent likely queries chemical databases and literature. Threats include data poisoning of chemical properties or synthesis pathways, which could lead to failed or highly dangerous physical reactions.

The framework orchestrates 13 expert-designed tools for synthesis and drug discovery. A major threat is insecure tool integration or tool misuse, where malicious inputs manipulate tool arguments to execute unintended or unsafe chemical calculations and planning steps.

Not certain from the listing — as an open-source tool, deployment safety depends on the user. If deployed without strict sandboxing, the execution of chemistry tools or APIs could lead to local system compromise or unauthorized network access to laboratory hardware.

Not certain from the listing — there is no mention of built-in safety guardrails, real-time monitoring, or logging of synthesis plans. This creates a blind spot where malicious or unsafe chemical recipes can be generated without detection.

Not certain from the listing — the agent lacks explicit compliance controls regarding chemical export regulations, dual-use technology restrictions, or identity verification for users requesting sensitive synthesis plans.

Not certain from the listing — while designed as a standalone agent with tools, integrating ChemCrow into automated laboratory ecosystems (A2A) without human-in-the-loop validation could lead to physical execution of hazardous chemical syntheses.