The EU AI Act for healthcare organisations: what changes when you deploy an AI assistant
Risk classes, human oversight and documentation, explained without legal jargon.
A procurement lead at a regional health board wants to give her team an assistant that answers questions about environmental criteria in tenders. A ward manager wants one that tells nurses which waste stream a product goes into. A sustainability officer wants help drafting the annual report. All three are reasonable requests. All three now come with a question that did not exist two years ago: what does the EU AI Act require of us if we switch this on?
The honest answer is: less than the headlines suggest for most of these uses, and more than nothing. This article walks through the parts of the regulation that matter when a healthcare organisation deploys an AI assistant, as opposed to building one, and what good practice looks like in each case.
Provider or deployer
The Act draws a line between the organisation that develops an AI system and places it on the market, the provider, and the organisation that uses it under its own authority, the deployer. A hospital that buys or commissions an assistant is almost always a deployer. The heavier obligations, such as conformity assessment, technical documentation and CE marking for high-risk systems, fall on providers. Deployers have their own, shorter list, which we come to below.
One caution: if your organisation substantially modifies a system, or puts its own name on it and changes its intended purpose, you can become a provider in the eyes of the law. Configuring an assistant, loading your own documents into it and setting your own rules does not normally cross that line. Retraining a model to make clinical decisions might.
The four risk tiers, and where an assistant usually sits
The regulation sorts AI systems into four tiers.1
Unacceptable risk. Practices that are banned outright, such as social scoring by public authorities or manipulative techniques that cause harm. These prohibitions have applied since February 2025. A sustainability assistant is nowhere near this category.
High risk. Systems listed in Annex III of the Act or embedded in products that already carry EU safety rules, such as medical devices. In healthcare the high-risk list includes AI used in emergency triage and dispatch and systems that decide access to essential services. It does not include an assistant that explains waste regulations or suggests tender wording, as long as that assistant is not making or materially shaping decisions about individual patients or individual people's access to care.
Limited risk, meaning transparency obligations. Any system that interacts directly with people must make clear that they are talking to an AI, and content generated by AI must be identifiable as such where it could mislead. These obligations apply from 2 August 2026. This is the tier most healthcare assistants land in.
Minimal risk. Everything else, with no specific obligations beyond general law.
For a knowledge assistant on sustainability, procurement or waste, the practical reading is: transparency obligations apply, high-risk obligations usually do not. The exception is a system that crosses into clinical decisions, patient prioritisation or individual access to services. If an assistant starts recommending which patient gets which treatment, it is a different system and a different conversation.
What the timeline looks like now
The Act entered into force in August 2024 with a staggered timetable. Prohibitions and the duty to ensure AI literacy among staff have applied since 2 February 2025. Obligations for providers of general-purpose models applied from 2 August 2025. Transparency obligations for systems that interact with people apply from 2 August 2026.
The dates for high-risk systems moved in 2026. Following the Digital Omnibus package adopted in the summer, obligations for stand-alone high-risk systems under Annex III now apply from 2 December 2027, and for high-risk AI embedded in regulated products, including medical devices, from 2 August 2028.2 For a healthcare organisation deploying an assistant that is not high risk, the date that matters is August 2026, and it has already passed.
What a deployer has to do
Even outside the high-risk tier, three obligations shape how an assistant should be rolled out.
AI literacy. Since February 2025, organisations must ensure that the people who operate and use AI systems have a sufficient level of understanding of how the system works, what it can and cannot do, and how to interpret its output. For an assistant used by nurses, buyers and facilities staff, this means a short, role-specific induction rather than a legal briefing: what the tool draws on, how to read its sources, when to escalate to a person.
Transparency. People using the assistant must know they are interacting with AI. In practice, the interface says so, the answers show where they came from, and the organisation keeps a record of what the system is and who is responsible for it.
Human oversight, if the system is high risk. For high-risk deployments, the Act requires that oversight be assigned to people with the competence, training and authority to intervene, that the system be used in line with the provider's instructions, that input data be relevant and representative, that logs be kept, and that affected workers be informed. These are not yet binding for most assistants, but they describe what a careful deployment looks like anyway, and they are what a public buyer will increasingly ask for in a tender.
What this means for the design of an assistant
The regulation sets the floor. Three design choices determine whether an assistant is trustworthy in daily use, and they map directly onto the obligations above.
Answers come from documents, not from the model's memory. An assistant that retrieves from a curated, validated knowledge base and cites the source of every answer gives staff something they can check and gives the organisation a defensible record. An assistant that generates freely from a general model gives neither.
The system says when it does not know. Confidence should be visible. Low-confidence answers should route to a person, not be delivered with the same fluency as high-confidence ones. This single feature does more for oversight than any policy document.
Every deployment has a file. Purpose, risk classification with reasoning, data sources, hosting location, named owner, the literacy material given to users, and a log of changes. If the system is ever reclassified, or a regulator asks, the file is the answer. If it is never asked for, it has still forced the right questions at the right time.
Two questions worth settling early
Where does the model run? The Act does not require EU hosting, but data protection law and your own information governance may. Healthcare organisations that want to keep patient and staff data under European jurisdiction should ask providers whether the model and the retrieval layer can run on EU infrastructure, including open-weight models hosted in the EU. Treating this as a procurement requirement is simpler than discovering it as a compliance gap.
Who is the provider? If an assistant is built for you by a software partner, the contract should say who holds provider obligations, who maintains the technical documentation, and what happens if the intended purpose changes. Clear roles now prevent an uncomfortable conversation later.
None of this is exotic. It is the same discipline healthcare already applies to medical devices and information governance, extended to a new category of tool. Organisations that approach it that way tend to find that the AI Act asks them to write down things they should have known about their own systems anyway.