Picture the first artificial intelligence sentenced to prison. The judge clears his throat and hands down five years for gross negligent manslaughter. The prison service must then work out whether the convict should be installed on a secure server in a secure facility, whether switching it off at night amounts to solitary confinement, and whether deletion is execution. The defendant offers no remorse, remorse not having been included in the enterprise licence. Civilisation glances briefly at its shoes and moves on.
The scene is absurd, but the problem underneath it is not. Artificial intelligence is moving from amusing office assistant to active participant in the world. It will not merely draft emails and summarise meetings. It will drive cars, approve loans, triage patients, translate contracts, assess benefit claims and control machinery that can injure, ruin or kill. When such systems go wrong, our legal instincts reach for the familiar remedy: find the responsible human. The difficulty is that there may no longer be one in any meaningful sense.
Civil law will probably cope first, because civil law is good at turning tragedy into invoices. If an AI mistranslates a contract, refuses a valid benefit claim, approves a disastrous loan or causes a collision, someone, manufacturer, operator, insurer, public authority, can be made to compensate the victim, and there is nothing conceptually impossible about AI insurance or professional indemnity. If the machine causes damage, a cheque can be made to come from somewhere.
Criminal law is the harder case, because criminal law is not merely about compensation. It is about guilt. It asks not only who pays? but who did wrong?, and that second question becomes awkward when the immediate actor is neither a person nor a passive tool, but a system making consequential decisions in the world. The AI may have no soul, no intention and no moral understanding, yet it can still classify, prioritise, recommend, refuse and kill. The ancient and convenient distinction between the person and the instrument begins to creak under the weight.
The autonomous car makes the strain obvious. When a human driver kills someone, we know roughly which questions to ask. Were they drunk, reckless, tired or distracted? Did they fail to brake, or ignore a sign? The law has had more than a century to build its habits around human error on the road. But suppose the car is driving empty across town to collect its owner; or suppose it is carrying only a child, a pet, a sleeping passenger; and suppose it kills someone. There is then no driver in the ordinary sense, no hand on the wheel, nobody present who can meaningfully be said to have committed the act. The machine was not merely being used. It was acting.
This is not a thought experiment: we have already held the trial. In March 2018, in Tempe, Arizona, a self-driving Uber test vehicle struck and killed Elaine Herzberg as she wheeled a bicycle across an unlit road. The vehicle’s systems detected her several seconds before impact but failed to respond properly; Uber had disabled the Volvo’s built-in emergency braking while testing its own system; and a safety driver, Rafaela Vasquez, sat in the front seat, nominally ready to intervene, while looking at her phone. Arizona prosecutors found no basis for criminal liability against Uber. Vasquez was charged with negligent homicide and, in 2023, pleaded guilty to endangerment. She was the only human being to stand in any criminal jeopardy at all. That is the model in miniature: the institution makes the choices that produce the risk, and the comparatively powerless human in the seat takes the conviction.
Even where an adult is at the wheel, the human fallback is largely a fiction. Emergency reactions are a matter of habit and posture, not reasoning: braking for a child in the road must happen before thought, and only happens if the human is watching, hands and feet ready, accustomed to intervening. A car that drives safely for months and errs once a year will not keep its occupant in that state; they will be reading, dozing, or simply elsewhere in their mind. A person who has not been driving does not become a competent driver in the half-second the machine hands back the wheel. Crash investigators have a name for the result: automation complacency. The system trains the human not to pay attention, then relies on the human to save it when attention is suddenly required.
What, then, would it take to keep that human ready? The car would have to err often enough to keep the driver alert, drift slightly, hesitate wrongly, demand the occasional intervention, not in order to drive better, but so that the fallback does not decay into theatre. That is plainly insane. A safety system should not be made deliberately worse to preserve the fiction that a human is in control. Either the car is autonomous, in which case responsibility cannot rest on a disengaged passenger, or it is not, in which case the revolution is merely cruise control with extra paperwork.
This is why the old slogan, guns don’t kill people; people kill people, belongs to the age of simple tools. A gun does not walk down the street, select a target and fire; a knife does not leave the kitchen, navigate the traffic and stab a passer-by. Such things are dangerous precisely because a human wields them. AI-driven machines are categorically different: they perceive, classify, select and act without a human directing each step. They may not be conscious, guilty or morally responsible, but they can be the immediate cause of a death all the same. The machine is not a murderer. Neither is it merely a hammer. We have built a third thing, and our vocabulary of blame has no word for it.
Nowhere is this third thing being built more deliberately than on the battlefield. The drones over Ukraine increasingly use autonomous or semi-autonomous terminal guidance: a human operator may select or confirm a target, after which the drone continues the attack even when jamming cuts the link. Both sides are plainly interested in pushing that autonomy further. We need not reach Star Trek’s Arsenal of Freedom, a weapon still dutifully slaughtering long after its makers are dead, to see the problem. A machine built to kill without a hand on it will, in time, kill the wrong person, and leave no hand to blame.
Hacking sharpens the problem further. When malevolent people seize an autonomous car and drive it into a crowd, the hackers are plainly criminals, but so, arguably, may be the manufacturer who shipped insecure software, the operator who monitored it feebly, or the regulator who licensed it at all. The more complex the system, the more tempting it becomes to point at the nearest available human and call that accountability.
The same temptation appears wherever the work is automated. If an AI prescribes the wrong drug, do we blame the doctor expected to rubber-stamp hundreds of recommendations a shift? If an AI mistranslates a legal document, do we ruin the translator told to “review” two hundred pages by lunchtime? If an AI helps design a roof or a bridge that later fails, does the engineer who signed the summary go to gaol, when the whole purpose of the system was to replace days of manual checking with minutes of automated output? If a warehouse robot crushes a worker after a software update, do we charge whoever happened to be standing nearest? In each case the human carries the liability precisely because the human has been stripped of the time, the knowledge and the authority that liability is supposed to presuppose.
These are not hypotheticals, and the pathology is older than the technology that will perfect it. Britain spent years prosecuting more than nine hundred sub-postmasters on the evidence of the Post Office’s Horizon software, not AI at all, merely buggy code that conjured phantom shortfalls, ruining lives, causing bankruptcies and being linked to multiple suicides, because the system was presumed reliable and the humans were obliged to disprove it from the dock. The Netherlands managed a related harm with algorithmic assistance: a risk-classification system and a brutal administrative machine branded tens of thousands of families as cheats, wrongly accusing many, separating children from parents and bringing down the Rutte government in 2021. If we did all this with dumb software, AI only removes the last excuse. Conventional code can at least in principle be audited by someone; a black-box model may not even offer that comfort.
Notice what these episodes share. Each was an institution profiting from automation, in money, staff or speed, while the consequences of the machine’s errors were loaded onto individuals who had no real power over it: the sub-postmaster who could not audit Fujitsu’s code, the parent who could not interrogate the risk model, the caseworker who clicked the final button. Human responsibility is real only where the human has meaningful control, and that is the whole of the matter. Responsibility must follow power, knowledge and control, not ceremonial proximity. A supervisor should be answerable only where the supervision is genuine: where they have the training, the time, the information and the authority to challenge and halt the process, and the institutional protection to do so without being punished for inefficiency. Where those conditions are absent, responsibility must travel up the chain to those who designed, bought, deployed and profited from the thing.
This is the danger concealed inside the comforting phrase human in the loop. It sounds prudent, and it can be a fraud. A genuine human in the loop slows the system down, scrutinises the output and holds the power to stop it; that may be necessary in high-risk cases, but it also dissolves much of the benefit of automation. If every autonomous car needs a vigilant human escort, we have reinvented the rule that a man must walk ahead of the motor car waving a red flag, very safe, and rather missing the point of the motor car. If every AI decision must be fully rechecked by a human expert, the system may remain useful, but the promised speed and savings evaporate.
The alternative is worse: pretending a human has checked something when they have not. Call this responsibility laundering. An institution deploys an AI system, banks the speed and the headcount reductions, designs the workflow so that no human could realistically understand or override every decision, and then produces a nominal supervisor when something goes wrong. The organisation keeps the profit; the human keeps the liability. In polite circles this will be called governance. In plainer words, it is keeping someone about the place to be hanged.
The honest objection is that responsibility, once smeared across a global supply chain, may simply disappear. If the lethal model was trained in one country, its planning code generated by an AI in another, integrated by a third and deployed by a firm in yours, then every node can point at the next, and three of them may lie beyond the reach of your courts. Worse still: what if the fatal error was written not by a foreign engineer but by another machine? It becomes tempting to conclude that nobody is culpable at all, that it is artificial intelligences all the way down.
But that conclusion is not a discovery; it is the defendant’s fondest wish. The dispersal of an act across many automated layers is no evidence that no one chose it. Someone, with a name and a salary and a fiduciary duty, decided to run this arrangement on a public road, or to put into a life-and-death setting code that no human had verified, and that decision is the culpable act, however many models sit downstream of it. The length of the chain is not a mitigation; it is the measure of how far the laundering has been allowed to run. The answer is not to find more powerless people to blame, but to refuse the premise: a system too opaque for anyone to be held answerable is not thereby excused. It is a system that ought not to have been let loose in that form.
There are, in truth, only four honest options. First, the machine is very nearly perfect, in which case the human becomes inattentive, deskilled and useless in the rare emergency. Second, the machine makes enough mistakes to keep the human sharp, in which case the human may stay competent but the benefits of automation collapse. Third, we take the human out of the loop and admit the system is genuinely autonomous, in which case responsibility must rest with the institution that built, sold, deployed and profited from it. Fourth, we decline to use the technology in that domain, at least for now. What we must refuse is the fraudulent fifth option, in which the machine does the work, the institution takes the savings, and a powerless human takes the blame. We must refuse it with particular firmness, because it is not a future risk. It is the current default.
This may be one of the bleakest futures of work: not a world in which people remain employed because their judgement is indispensable, but one in which they remain employed because somebody must still be available to be sued, struck off, prosecuted or gaoled when the algorithm misbehaves. The work is automated; the punishment remains manual. A post whose chief function is to go to prison when software errs is not a profession. It is a hostage situation with pension contributions.
The better principle is a very old one: cui prodest? Who benefits? If a bank saves money by automating its lending, it should answer for the lending; if a hospital automates its triage, it should answer for the triage; if a manufacturer sells an autonomous machine, it should answer for the autonomy. The savings and the liability belong together. And a public authority that processes hundreds of thousands of welfare, immigration or policing decisions by machine must not be permitted to point at the exhausted caseworker who clicked confirm.
This will require new legal structures. High-risk AI systems can be made subject to registration, certification, mandatory insurance, tamper-proof audit logs, independent testing and named officers with real authority and real exposure; dangerous systems can be suspended, decertified or banned from particular uses; hacking can be treated as a central design obligation rather than an embarrassing edge case. We do not need to pretend that AIs are people in order to do any of this. We already hold corporations criminally and civilly to account, and a corporation is merely a legal ghost wearing a logo. The reason the Uber corporation walked away from Tempe while the woman in the seat did not is not that the law cannot reach institutions; it is that we have not yet decided to make it.
We will not, in the end, send AIs to gaol. There will be no prison wing for fraudulent chatbots, no parole board for reckless credit models, no gallows for a delivery drone that kills. But the absurdity points at a real and imminent choice. Either we build a legal order in which responsibility follows power, control and benefit, or we build one in which powerless humans are kept on the payroll to be punished for the decisions of machines and the institutions that own them.
The future should not consist of machines doing the work and humans doing the time.