ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage

Because the problem is not the All-Seeing Eye. It is the highway that blocks itself.

For years we have been reporting on artificial intelligence through the eyes. Big Brother. The Eye of Sauron. Tolkien’s palantír – the seer stone that shows everything, filters everything, subjugates the beholder. Beautiful metaphors. Powerful. Culturally honest. And perhaps already insufficient. Not because they are wrong. But because they describe a problem we are leaving behind without realising it. Someone is using them to comment on Pope Leo’s beautiful encyclical, and perhaps Palantir is not quite the appropriate metaphor, let us try to reason.

Tolkien still had a legible enemy. In The Lord of the Rings, evil is narratively comprehensible. There is a tower. There is an Eye. There is a will. The palantír show a filtered reality – but filtered by someone. Saruman believes he is using the tool and is being used by it. Denethor sees only what Sauron chooses to show him.

The pattern is clear: he who controls the flow of vision controls the beholder.

It is a perfect metaphor for 20th century surveillance. Someone collects the data, someone interprets it, someone governs. There is still a centre. There is still a Sauron with a plan and a building with a sign. Forty years later, William Gibson had already realised that things were changing shape. Tolkien still had castles. Gibson already had suburbs. In Tolkien’s world, evil inhabits a tower. In Gibson, evil is traffic.

Gibson: when the system loses its centre. Neuromancer came out in 1984. Gibson invented cyberspace and defined it as follows: a consensual hallucination experienced by billions of operators every day.

Not a system of control. A collective hallucination that no one governs and everyone inhabits. And Wintermute – the novel’s AI – does not pursue human goals. It pursues emergent goals that even its creators do not fully understand. It is not evil. It has no agenda. And that is precisely the disturbing point.We are culturally prepared for tyrants. Much less so for emerging systems. It has a direction that no one has deliberately chosen. Gibson wrote in 1984 and described 2026 more accurately than Tolkien can.

Because the problem is no longer the palantír – the instrument of vertical domination. The problem is the horizontal hallucination: the distributed system that produces real effects without a responsible centre.

The traffic jam that has no cause

In 2008, physicist Yuki Sugiyama and his colleagues at Nagoya University conducted a simple and disturbing experiment. Twenty-two cars on a circular track. Constant speed. No obstacles. No accidents. No external causes.

After a few minutes, traffic comes to a standstill.

A wave of braking spreads backwards like an echo. One car slows down a little, the one behind brakes a little more, the one behind brakes even more. The system collapses in on itself without anyone having wanted it, planned it, caused it.

Phantom traffic jams

Emerging systems without a director. Real effects without localisable responsibility. Consequences without intention. Contemporary AI is becoming something similar: not a central brain that decides, not an eye that watches, but a giant distributed cognitive traffic jam where effects are there, decisions happen, and the point of origin is untraceable.

The problem is not that the machines decide for us. It is that no one knows where the decision started anymore.

Zuboff: he doesn’t look at you, he extracts you

There is, however, an economic mechanism that feeds the gridlock from outside – and which Shoshana Zuboff has described with surgical precision in her The Capitalism of Surveillance.

Zuboff’s insight is not that we are being watched. It is more radical: our daily experience is mined as raw material to produce behavioural prediction products – sold to those who want to change our future choices.

Non-surveillance. Mining. Like a mine. Only the mine is you. The problem is not that you are being watched. It is that you have become raw material.

And the difference is not subtle. Surveillance is watching you. Extraction consumes you slowly, fragment after fragment of experience, without you feeling anything – because every single withdrawal is imperceptible.

From surveillance to dissolution

There is a phenomenon that physicists call hysteresis: the delay between a stimulus and the system’s response, which never returns exactly to the starting point. Every deformation leaves a memory. Every shortcut slightly alters the inner shape of the person taking it.

This is what happens from the inside, while Zuboff describes the mechanism from the outside.

Micro-choice after micro-choice – which path to take, what to read, how to respond – we delegate fragments of judgement to systems that do not understand the context but know the probabilities. Each delegation imperceptibly lowers the threshold below which we feel the need to really decide. It reduces inner friction. It eases the fatigue of judgement. And any cognitive comfort left there long enough ends up feeling natural. Even stopping to decide

Which is not a problem – until the moment you realise that without that effort you can no longer find your way around on your own. Technology does not just replace work. It risks progressively replacing responsibility. Not with an act of force. With a thousand consecutive little conveniences.

The junk dealer and the motorway

I confess one thing. I work with fragments. I take pieces of Kabbalah, of rock, of complexity, of mythology – and reassemble them into forms that I hope give meaning to the present. It is the junkman’s job: not to produce new materials, but to recognise which old materials illuminate something new.

Tolkien still illuminates something. But it illuminates the first season of the problem: that of surveillance, of the power that watches, of the centre that controls.

Gibson illuminates the second: the system without a centre, the collective hallucination that no one governs and everyone inhabits.

Tolkien was still narrating power. Gibson was already narrating the environment. Today, we live within ecosystems that no longer even need narrating.

Zuboff illuminates the mechanism: not the control, the extraction.

We are already in the third season. The one of silent dissolution – where no one is watching you, no one consciously extracts your critical sense – but the system, on its own, like a motorway without accidents, finds a way to come to a standstill.

Sauron at least had a plan. Wintermute at least had a direction. The phantom traffic jam has neither. It forms itself. It feeds itself. And no one, looking around, can understand why the highway is at a standstill.

The question that remains

The question is not whether AI makes decisions for us. It is whether, by the time it does, we still notice it. Or whether we have already stopped expecting to notice. Perhaps the real risk of artificial intelligence is not losing its job. It is slowly losing the very reflex of responsibility. Like motorists stopped in a phantom traffic jam, we keep braking, accelerating, imitating whoever is in front of us, no longer knowing who touched the pedal first.

And perhaps the real success of AI will come the day when no one will feel the need to understand why they are obeying.

Beppe Carrella
WRITTEN BY Beppe Carrella

©2025 Fondazione per la sostenibilità digitale

Tech Economy 2030 è una testata giornalistica registrata. Registrazione al tribunale di Roma nr. 147 del 20 Luglio 2021

Powered by DTILab  - Designed by Fattoria Creativa - Developed by EHT