The experiment

An AI agent instead of a human colleague? Here’s how the team’s work is changing

Attributing a human role to AI agents reduces the perception of personal responsibility and may undermine managerial control. The experiment conducted by the Boston Consulting Group and Boston University

by Gianni Rusconi

6 October 2026

Aggiungi Il Sole 24 Ore
ai preferiti su Google

6' min read

Translated by AI

Versione italiana

6' min read

Translated by AI

Versione italiana

A name, a role, perhaps a position on the organisational chart. All it takes is to turn an artificial intelligence agent into a ‘company employee’ to change the way people relate to its work? The answer, drawn from an experiment conducted by Boston Consulting Group and Boston University, suggests so – and in a direction that is far from obvious.

The risk of having less control

The study involved 1,261 managers working in human resources and finance in the United States, Canada and the European Union, and presented participants with documents containing the same errors, randomly attributed to an artificial intelligence tool, a human colleague or an ‘AI employee’. When the author was presented as a team member, managers identified 18 per cent fewer errors than when the document was presented as having been produced by a technological tool, and requested a further review in 44 per cent of cases. Furthermore, the personal responsibility attributed to the error fell by nine percentage points, whilst that attributed to the system rose by eight points.

The result of the experiment is particularly interesting because the idea of assigning a name and an identity to agents stems precisely from the desire to make them more familiar and to facilitate their integration into work teams. However, there is a risk that this approach could have the opposite effect in terms of accountability – a risk highlighted by Roberto Ventura and Nicolò De Benetti, Managing Director & Partner and Principal respectively at BCG in Italia. ‘Humanising agents to an excessive degree,’ the two managers explained to *Il Sole 24 Ore*, ‘can create the risk of losing control over them in their role as tools; given the current level of technological sophistication, when a task is ambiguous and the path to the result is unclear, it is impossible for the agent to become fully autonomous.” A clear picture that highlights one of the most important crossroads in the process of adopting AI, namely the need to ensure that portraying agentic tools as team members does not end up creating a sort of ‘grey area’ in which human control weakens just as the autonomy of the software-driven system increases.

The AI agent is already at work, but is not yet an employee

There is no doubt that AI agents are beginning to be integrated into business processes in a tangible way, although, according to BCG experts, adoption has not yet reached scale and the inconsistent structure of processes currently limits their scope of application. The difference, they emphasise strongly, is particularly significant between large companies and small and medium-sized enterprises: in the former, some ‘agent-driven’ processes are already in place, whilst in the latter, artificial intelligence is more often incorporated into everyday applications rather than being used as a standalone solution to solve specific problems. In this sense, the shift towards agents represents a paradigm shift: following the phases of machine learning and predictive AI, and the more recent phase of generative AI, companies’ focus is generally shifting towards systems capable of carrying out sequences of tasks with an increasing degree of autonomy. ‘Agents,’ Ventura and De Benetti confirm, ‘are good at handling unambiguous workflows, where the input and output are clear and the result is verifiable. A concrete example comes from call centres, where AI agents are becoming established in roles where the level of responsibility is relatively low.”

From this perspective, the replacement of human staff does not take on the central importance often attributed to it. The issue also concerns what part of the work will remain for people once the agent takes on an increasing proportion of the tasks, thereby boosting productivity and removing some of the bottlenecks caused by a lack of resources. This is the case in software engineering, where the ability to produce outputs rapidly can increase the volume of work an organisation is able to handle. Human specialists could therefore broaden their scope by focusing more on setting objectives, verifying results and making decisions at the most complex stages. Rather than hiring a ‘digital employee’ – as the two BCG managers suggest – companies are beginning to ‘agentise’ individual workflows, repeatable tasks or even processes where the outcome can be defined and verified. However, it is still premature to speak of agents fully integrated into organisational charts or of actual organisational charts built around agents, with the exception of a few start-ups experimenting with organisational models akin to a ‘black factory’ populated by robots.

Where it can replace humans and where human judgement is still needed

But if the agent is not yet an employee, what tasks can it nevertheless gradually take over from human labour? When asked this specific question, the answer cannot be framed simply in terms of professions, as the decisive selection criterion, according to Ventura and De Benetti, is above all the ambiguity of the process. ‘Checking invoices, or data entry and call-centre work, is one thing; devising a marketing strategy is quite another, precisely because in the latter case a decision-making component comes into play which still represents a distinguishing factor today.’ Whilst it is therefore reasonable and possible to envisage the gradual automation of more structured and verifiable tasks, in roles requiring interpretation, judgement and the setting of objectives, the human role remains more difficult to replace, even though the boundary between the two scenarios is by no means static. Response speed and output quality are the cornerstones of this agent-driven transformation of work, and the factors that will make the difference will be, on the one hand, the evolution of the systems’ reasoning capabilities and, on the other, the volumes of work that agents will be able to generate – without, of course, forgetting the economic benefits of their introduction. Broadly speaking, the organisation of the future may not be built simply by adding ‘AI employees’ alongside human ones, but by redesigning processes around a variable combination of people and agents.

Accurately forecasting the impact of these factors on the organisation over the next three to five years remains difficult in any case because, as the BCG experts emphasise, ‘the pace of technological change is exponential’. By contrast, the impact on digital activities carried out on computers appears more predictable, at least in the short term, whilst so-called ‘physical AI’ – that is, artificial intelligence embedded in systems capable of acting in the physical world – will probably remain more closely linked to experimentation than to large-scale adoption.

If the officer makes a mistake, who is held responsible?

Finally, on the subject of liability, the message that emerges from the BCG-Boston University study is clear: software cannot be held liable in the strict sense and, above all, cannot be made the scapegoat for a wrong decision. In other words, if an agent produces an incorrect output, one cannot claim that ‘the AI got it wrong’. And this is precisely the paradox revealed by the experiment: when artificial intelligence is presented as a member of the team, personal accountability tends to diminish, whilst the responsibility attributed to the system increases; as a result, errors become more difficult to detect.

For businesses, the priority is therefore to establish a chain of accountability before increasing the level of autonomy granted to agents. ‘We would never advise any company,’ emphasise Ventura and De Benetti, ‘to rely entirely on agent technology, which is, after all, still just software. If anything, we would suggest they put safeguards in place to limit potential errors and establish a governance framework capable of coordinating the use of these systems.’

Returning to the issue of assigning responsibility in the event of an error, it is clear that the party to be held accountable cannot be a single actor within the ecosystem but rather several parties: namely, those who design the AI agent, those who use it and those who govern it, subject to the definition of the various levels of responsibility that will be established by regulatory and case-law developments. The real issue regarding AI agents in organisations, therefore, does not seem to be whether to regard them as colleagues or tools; the real challenge is to understand which parts of the work should be entrusted to them, with what degree of autonomy and under what level of human oversight. To achieve this, businesses must develop the capacity to simultaneously redesign technology, processes and human work in order to clearly establish who does what, who oversees the process and, above all, who is accountable when things do not go as planned. Simply transforming the AI agent into a ‘employee’ in every sense of the word, at least at present, risks being nothing more than an organisational metaphor.