The AI Act and healthcare: what obligations for your AI systems?
Most AI systems in healthcare fall into the “high-risk” category. An overview of the obligations and of the right reflexes to adopt from the design stage.
The European Artificial Intelligence Regulation (AI Act) places most AI systems used in healthcare in the “high-risk” category, which carries the most demanding obligations.
Documentation and oversight
Risk management, quality of training data, technical documentation, human oversight and robustness: all requirements to be built in from the design stage rather than retrofitted.
How it fits with the GDPR
The AI Act does not replace the GDPR: it comes on top of it. A data protection impact assessment remains necessary for the processing of health data, and the legal basis must be secured.
The risk pyramid, applied to healthcare
The AI Act works in tiers. At the top, prohibited practices (manipulation exploiting vulnerability, social scoring, certain biometric identification) — rarely at issue in healthcare, but worth checking for behavioural “wellness” tools. Next, high risk, which captures most clinical AI through two doors: Annex I for AI that is a safety component of a medical device subject to the MDR or IVDR (diagnostic support, imaging quantification, embedded algorithms), and Annex III for certain uses such as emergency call triage or assessing access to essential services. Then simple transparency (chatbots, generated content) and, finally, minimal risk — the majority of office uses.
Qualification is the decisive step: it determines everything else, and it must be documented. A single organization almost always houses all four tiers at once.
Provider or deployer: two sets of obligations
The provider (the party that develops or places the system on the market) carries the heaviest load: a risk management system throughout the lifecycle, governance of training data, technical documentation, logging, robustness and cybersecurity, user information, marking and registration.
The deployer — the facility or professional that uses the system — is not off the hook either: use in accordance with the instructions, effective human oversight (trained professionals with the right and the means to overrule the machine), control over the quality of input data, information of the persons concerned, and for certain public bodies a fundamental rights impact assessment. All of this dovetails with the GDPR’s DPIA, which remains due whenever health data is processed on a large scale.
Where the timeline stands (to date)
Update — summer 2026: the timeline is now settled. Prohibitions and AI literacy have applied since February 2025; the regime for general-purpose models since August 2025; transparency obligations and the governance framework since August 2, 2026. The postponement of the “high-risk” obligations is enacted by Regulation (EU) 2026/1744, published in the Official Journal on July 24, 2026: Annex III is pushed back to December 2, 2027 and Annex I (AI embedded in medical devices) to August 2, 2028. For the details of what applies right now, read our analysis of the new timeline. One principle remains: never confuse a postponement with an exemption.
Case studies: how the AI systems we meet most often are classified
Diagnostic support in imaging (nodule detection, quantification): a medical device, hence high-risk via Annex I — MDR and AI Act documentation working together. The triage or appointment-booking chatbot: transparency required, and vigilance as soon as it touches on medical advice, because the line into medical-device territory is quickly crossed. The consultation transcription and summary tool: general-purpose model regime on the provider side, reinforced GDPR on the facility side — DPIA, patient information, retention periods for recordings. AI for coding procedures or optimizing schedules: depending on purpose, limited risk or Annex III — a written qualification is your first protection. “Consumer” generative AI used by staff: the real issue is health data leaking into uncontrolled tools — an acceptable-use policy and secure alternatives are a must.
Your six-step roadmap
1. Inventory every use of AI, official and unofficial — shadow AI is the rule, not the exception. 2. Qualify each system in writing (prohibited, high-risk Annex I or III, transparency, minimal) with its justification. 3. Prioritize by actual risk: what touches care and health data comes first. 4. Upgrade supplier contracts: AI Act guarantees, GDPR processing terms, reversibility, logging. 5. Organize human oversight: who can overrule the machine, how, with what audit trail. 6. Train — the AI literacy obligation has applied since February 2025 — and revisit the inventory with every new project.
This is not one more “compliance” project: it is the precondition for deploying AI with confidence, and a decisive commercial argument in front of increasingly demanding hospital purchasers.