An AI system does not always behave in operation as it did in testing. Data drifts, users use the system differently than intended, errors only show in edge cases. The AI Act therefore requires providers of high-risk systems not to look away after placing them on the market.
The monitoring system
Under Article 72 of the AI Act, providers establish a post-market monitoring system proportionate to the nature and risks of the system. It actively and systematically collects data on the system’s performance throughout its lifetime, documents and analyses it, and checks whether the system continues to meet the requirements. The basis is a monitoring plan forming part of the technical documentation; the Commission sets a template for it.
Serious incidents
If a serious incident occurs, the provider must report it to the market surveillance authority of the member state concerned (Art. 73). The basic deadline is 15 days after becoming aware. For a widespread infringement or a serious incident in critical infrastructure it shortens to two days, and in the event of a death to ten days. Where necessary, an incomplete initial report is allowed, to be completed later.
The deployers’ role
Deployers monitor operation according to the instructions for use and inform the provider when they identify risks or a serious incident occurs. Without that feedback the provider’s monitoring is blind. Contracts between provider and deployer should therefore set out which data flows back and how fast.
Technical implementation
Monitoring is first a question of observability: logs of inputs and outputs to the permitted extent, metrics on error rates and deviations, user feedback, an escalation process. For generative components, ongoing evaluation against a fixed test set belongs to it, so that degradation after model changes is noticed. Data protection limits what may be logged; the answer is deliberate selection, not abstention.
Timing
These obligations are part of the high-risk requirements which, following the Digital Omnibus, apply to Annex III systems from December 2027. Anyone developing such a system should plan monitoring into the architecture from the start; retrofitted, it becomes more expensive and patchier. Our high-risk conformity offer covers this part.