EU AI Act: Draft Commission guidelines on classification of high-risk AI systems

This section sets out the meaning of AI system and guidance to understand a system’s intended purpose, which is key to deciding whether it is classified as high-risk. It explains the importance of the information contained in the provider’s promotional and sales materials, user instructions and technical information. These materials must clearly outline the AI system’s functions, specifying its scope and intended purpose. If the AI system is designed for multiple uses, all of these uses should be listed to clarify whether any application could result in a high-risk classification.

Including a disclaimer in the terms and conditions stating that high-risk uses are not allowed will not be enough to prevent a system from being classified as high-risk. If the provider’s marketing, examples, or product descriptions imply or encourage high-risk uses, the AI system may still be considered high-risk. 

Annex I provides a practical methodology for determining when an AI system is considered high-risk under Article 6(1) of the AI Act. This applies to AI systems that are either:

• products themselves; or
• safety components of products, where those products are subject to specific EU harmonisation legislation (e.g. relating to machinery, toys, lifts, medical devices, or vehicles).

There are two cumulative conditions for an AI system to be classified as high-risk:

• the AI system must be either a regulated product or a safety component of such a product; and
• the product (or safety component) must be subject to a third-party conformity assessment under the relevant harmonisation legislation.

The Annex further explains the meaning and scope of safety function, which governs whether an AI system falls within Annex I and includes preventive and mitigation functions. The guidelines distinguish safety functions from optimisation and quality control functions.

The largest section of the guidelines covers AI systems in the eight areas listed in Annex III. This article focuses on biometrics, critical infrastructure and employment, but Annex III also covers education and vocational training, essential services, law enforcement, migration and administration of justice.

The section starts with some principles applicable to all these areas, including:

• The role of human involvement, which, because it cannot change the purpose and area in which a system is intended to be used, has no effect on the classification of the system as high-risk, but is a requirement for compliance with the rules for high-risk AI systems. In simple words, if the AI system’s intended purpose falls within Annex III, a human intervention does not, by itself, bring it out of scope. The guidelines remind that human oversight is required by Article 14 of the AI Act and is essential for all high-risk AI systems.
• The filter mechanism set out in Article 6(3) of the AI Act, which, subject to some exceptions, allows providers of AI systems listed in Annex III to exempt those systems from high-risk classification, even though the conditions for such classification have been met based on the intended purpose of the system. The guidelines explain the four alternative conditions which state that the provider can rely on the filter mechanism where the AI system is intended to: (i) perform a narrow procedural task; (ii) improve the result of a previously completed human activity; (iii) detect decision-making patterns or deviations from prior decision-making patterns and is not meant to replace or influence the previously completed human assessment without proper human review; or (iv) perform a preparatory task to an assessment relevant for the purposes of the use cases listed in Annex III. The guidelines are clear that these exemptions must be interpreted narrowly, as Article 6(3) of the AI Act provides for exemptions from the general rules aimed at protecting fundamental rights.

This section of the guidelines goes on to provide more details about each of the areas specified in Annex III, providing examples of AI systems falling within and outside each high-risk use case. Here are our key takeaways.

Annex III provides that the following types of AI system are high-risk:

• Remote biometric identification (RBI) systems
• AI systems intended to be used for biometric categorisation, according to sensitive or protected attributes or characteristics based on the inference of those attributes or characteristics
• AI systems intended to be used for emotion recognition 

The guidelines explain how biometric data can be extracted from a living individual’s physical, physiological or behavioural attributes and provide examples, including less common examples such as the geometry of veins in their hands. They highlight the interplay between the high-risk systems specified in Annex III and systems prohibited under Article 5 of the AI Act:

• Real-time RBI systems in publicly accessible spaces for the purposes of law enforcement
• Systems that categorise individuals based on their biometric data to deduce or infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation
• Emotion recognition systems in the workplace and education institutions

Annex III of the AI Act lists as high-risk six use cases of AI systems intended to be used as a safety component in the management and operation of specified cases of critical infrastructure: critical digital infrastructure, road traffic, and the supply of water, gas, heating or electricity. The guidelines explain when an AI system should be classified as a safety component, distinguishing this from a cybersecurity component, ie an AI system used for cybersecurity purposes. This assessment must focus on whether the AI system directly protects the physical integrity of the infrastructure by reducing, preventing, controlling or mitigating risks that could lead to physical harm to people or physical damage to property. 

Annex III of the AI Act lists as high-risk two use cases of AI systems intended to be used in employment, management of workers and access to self-employment:

• AI systems intended to be used for the recruitment or selection of individuals, in particular to:
  
  • place targeted job advertisements
  • analyse and filter job applications (including screening CVs)
  • evaluate candidates (including any assessment of a candidate’s (including self-employed) suitability, merit or potential)
• AI systems intended to be used in the context of employment, ie to:
  
  • make decisions affecting terms of work-related relationships, promotions or termination of work-related contractual relationships
  • allocate tasks based on individual behaviour or personal traits or characteristics
  • monitor and evaluate the performance and behaviour of persons in work-related relationships

The guidelines explain that the key factors which determine when these AI systems are high-risk are whether the systems could limit access to employment opportunities or lead to a risk of discrimination. This section also sets out when the filter mechanism referred to above could apply to AI systems used in employment. This part of Annex III is likely to be the most important category for most of our clients, particularly given the broad use of various AI tools in HR processes.

While the guidelines discussed in this article address the AI Act as it is currently in force, subject to the deadlines extended by the AI Omnibus, on 22 May the Commission published a report on its work to assess whether there is a need to amend the list of prohibited AI practices and high-risk use cases. The Act requires the Commission to conduct this assessment once per year. The report concludes that, because the rules on the enforcement of the chapter on prohibited AI practices will not enter into force until 2 August 2026 and the guidelines on the classification of high-risk AI are still under preparation, it would be premature to make amendments at this time. However, the Commission identifies a number of issues to monitor, including AI systems intended to develop or distribute malware, therapy chatbots, AI systems enabling dark patterns and addictive design, and proposals to broaden the prohibition on emotion recognition technologies and the scope of the critical infrastructure use case to include systems integral to infrastructure reliability and continuity. Concerning CSAM (child sexual abuse material), the Commission acknowledges a potential regulatory gap for AI systems generating CSAM and non-consensual intimate content. However, a more substantive evaluation is deferred until there is further practical experience and implementation data.