What is a conformity assessment under the EU AI Act and how do I complete one?

A conformity assessment under the EU AI Act is a process to ensure an AI system complies with the Act's requirements, and it can be completed by establishing and maintaining an AI Management System (AIMS).

To complete a conformity assessment, an organization should:

• Define the context and scope of its AIMS by determining internal and external factors, interested parties, and the boundaries of the AIMS.
• Demonstrate leadership by establishing an AI policy, documenting the AIMS, and assigning clear roles, responsibilities, and authorities for AI governance.
• Plan for risks and opportunities by implementing an AI risk assessment and AI impact assessment process, and setting AI objectives. This includes identifying potential positive and negative impacts to individuals, groups, and society, as well as data sensitivity and regulated data exposure.
• Provide necessary support through resources, competence, awareness, communication, and control over documented information to operate the AIMS effectively.
• Implement operational planning and control by executing AI risk and impact assessments and operating AI systems under defined controls. This also involves maintaining an inventory of AI systems, including models, agents, tools, and data flows.
• Evaluate performance through monitoring, measurement, analysis, and evaluation of the AIMS, including internal audits and management reviews. This should include evaluating AI system security and resilience, such as adversarial robustness and prompt-injection resistance. Mechanisms should also exist to log decisions and trace AI behavior.
• Commit to continual improvement by addressing nonconformities and enhancing the AIMS over time.
• Address third-party risks by having policies that cover risks from third-party models, datasets, and tools, tracking provenance, licensing, and model-update risks. This maps to OWASP LLM03/LLM05 (supply chain).

The provided sources do not specify a control number or clause for the EU AI Act itself, but rather describe the components of an AI Management System that would support such an assessment.