constitutional-ai (AI-Research-SKILLs) — agentic threat model

The agent interacts directly with foundation models to perform self-critique and alignment. The primary threat is adversarial prompt injection that bypasses the constitutional rules, leading to misaligned training outputs or model reprogramming during fine-tuning.

The agent manipulates training and fine-tuning datasets. A key threat is training data poisoning, where malicious inputs are injected into the self-critique loop to systematically bias or backdoor the resulting model.

The agent framework orchestrates the self-critique loop and generates training code. Vulnerabilities here include insecure tool integration, where the code-generation engine can be manipulated into executing malicious payloads under the guise of training scripts.

Because the agent writes and runs training code, the underlying deployment infrastructure is highly exposed. Without strict sandboxing, container escape, privilege escalation, and unauthorized resource consumption during training runs are critical threats.

The agent acts as an evaluation and guardrail mechanism itself. The primary threat is evaluation gaming, where adversarial training samples bypass the constitutional critique phase undetected due to blind spots in the critique prompts.

Not certain from the listing — there is no explicit mention of identity management, access control, or compliance auditing for the code execution environment. General risk of unauthorized users triggering training runs.

The agent is described as a sibling to LlamaGuard/NeMo-Guardrails skills. In a multi-agent ecosystem, a compromise of this agent could allow it to feed poisoned guardrail configurations or corrupted alignment policies to downstream agents, causing cascading failures.