From empathetic assistants to deepfakes: the new transparency rules for AI

In the age of AI, it is becoming increasingly difficult to distinguish the original from the artificial, the real from the fake: text, images and audio created, modified or reproduced using AI are becoming ever more widespread. Take, for example, receiving a phone call. On the other end, a friendly voice, the click-clack of a keyboard, a pause at just the right moment. None of those details are real: the voice is synthetic, the keyboard is a sound effect, the hesitation is programmed. Within seconds, that voice has already worked out whether we’re in a hurry, whether we’re wary, and whether it’s worth pressing on. This is the point at which artificial intelligence ceases to be a tool and becomes a conversation partner. A conversational system recognises emotions and moods in real time, adapts its tone, and chooses the right approach to achieve its objective: selling a service, obtaining data, or closing a case. The system can store the information gathered and link it over time, guiding actions and behaviour. That voice may be anonymous. Or it may unlawfully mimic the voice of a family member asking for help or money: the news reports incidents of this kind almost daily, to the detriment of citizens and businesses.

The impact is even greater when the images are also computer-generated. The fake video of Ukrainian President Zelensky – in which, during the first weeks of the conflict with Russia, he called on soldiers and civilians to surrender – has gone down in history. Its poor quality limited its impact, but the precedent remains: technology affects the very certainty of social relationships.

But virtual agents aren’t used solely for illicit purposes. Quite the contrary. Anyone who has tried chatting with the leading chatbots – which also offer voice functionality – is well aware of the quality of these systems. They respond to everything; you can choose the tone and the approach. Often, their empathy surpasses that of humans. That is why virtual assistants are becoming increasingly widespread in the retail sector and are also starting to be used to support patients in the healthcare sector.

It is in this context that Article 50 of the AI Act, which sets out the so-called rules on the transparency of AI-generated content, applicable from 2 August 2026. The obligations operate on two fronts. Providers must design systems intended to interact directly with people in such a way that the user is informed that they are dealing with a machine, and must ensure that the generated content is recognisable as artificial. Deployers — those who use these systems for professional purposes — have their own obligations: these include informing people exposed to emotion recognition or biometric categorisation systems. The Regulation pays particular attention to the latter, because knowing how an interlocutor feels – and presumably what they are thinking – places them in a vulnerable position and paves the way for uses that Article 5 expressly prohibits, such as the manipulation of behaviour.

The European legislator is aware, however, that transparency must not amount to a pointless formality, and has provided for significant limitations. A privacy notice is not required when the context leaves no doubt as to the artificial nature of the system: this is the case with the support chatbot within an e-commerce app. It is not required when AI performs a function that is merely ancillary to the creation of the content, such as editing. It is not required in the cases provided for by law, such as its use for the purpose of combating crime. There is also an exemption for texts published with the aim of informing the public on matters of public interest; the obligation to disclose the artificial origin does not apply if the content is subject to human review and editorial responsibility is assumed for it.