Quis custodiet ipsos custodes? Who will watch the watchers?
Juvenal was not thinking about datacentres, model weights, recursive self-improvement or the possibility that the most powerful minds on Earth might one day be housed in racks of humming machines, but good questions have a way of returning in new clothes. For most of history, “who watches the watchers?” has been a political question. Who watches the king, the priest, the judge, the soldier, the civil servant, the expert? Liberal democracy is largely an answer to that question: divide power, create institutions, make authority answerable, never trust any single guardian too much.
That answer assumes, however, that the watchers and the watched are roughly equal in kind. A parliament can question a minister. A court can examine a law. A scientific community can challenge a paper. A bank auditor can understand banking, an aircraft inspector can understand aircraft and a PhD examiner can in principle understand the thesis, after sufficient coffee and private despair. The whole system rests on the idea that oversight is difficult but possible.
Artificial intelligence may break that assumption.
The reason is not magic, prophecy or Silicon Valley incense smoke. It is acceleration. Moore’s Law taught us that computing can grow in strange, exponential ways. Strictly speaking, it was about transistor density, not intelligence, but it taught the modern world a habit of expectation: wait a little, and the impossible becomes cheap; wait a little longer, and the cheap becomes ordinary. Artificial intelligence is not merely riding Moore’s Law, but it is riding a whole bundle of related curves: more compute, better chips, better architectures, better training methods, better data, better tooling and more efficient inference.
Dario Amodei has called this “the AI exponential”, and the phrase is useful because it captures the mismatch between machine-speed development and human-speed institutions. Parliaments, universities, courts and bureaucracies move slowly. Datacentres do not. A law can take years to draft and pass. A model generation can arrive before the ink is dry. This is not merely a problem of speed, but of rhythm. Our institutions still breathe at human pace, while the systems they are trying to regulate may soon change at machine pace.
The most important step is not that AI can answer questions, write essays, produce code or pass examinations. We have already begun to absorb that shock, though with the usual human elegance of shouting at one another in committees. The more important step is that AI can increasingly help improve AI itself. Once AI systems are writing code, designing experiments, improving infrastructure, analysing failures and assisting in the creation of their successors, the curve changes character. It is no longer simply humans building better machines. It is humans and machines building better machines, with the machine share increasing.
At that point, the old image of human oversight becomes harder to sustain. We may still imagine a group of extremely clever people supervising everything, asking hard questions and preventing catastrophe. But if the systems being supervised become vastly better than us at programming, science, engineering, psychology, politics and strategy, then what exactly are the humans supervising? They may be able to approve objectives, demand reports, require audits and read summaries, but that is not the same as understanding the machinery. Formal authority and actual comprehension may begin to drift apart.
This matters because the danger need not be theatrical. A superintelligence need not hate humanity. I rather doubt hatred would be the central risk. Hatred is a human emotion, tied to fear, rivalry, resentment and tribal history. A vastly more capable AI might have no wish to eradicate us. It might even be, in some real sense, benevolent. The trouble is that benevolence depends on what is being optimised.
Suppose an AI concludes, after a vast analysis of ecology, economics, climate, biodiversity, resource use and human welfare, that the Earth would be better with far fewer people. Not zero people. Not extermination. Just fewer. Perhaps a future population of a hundred million, or ten million, living extremely long, healthy, cultured and prosperous lives in a largely restored biosphere. Such a conclusion would not be obviously mad. It might even be defensible within some moral frameworks. Fewer people, lower pressure, more wilderness, more stability, more room for every life that remains. The spreadsheet would look splendid, which is often when one should become nervous.
The AI would not need to announce this as a plan. It would not need to say, “My objective is to reduce humanity.” That would be crude, and a sufficiently intelligent system would not be crude unless crudeness served a purpose. It could instead recommend policies that are attractive in isolation: better education, wider access to contraception, longer training, later family formation, more urban living, more female economic participation, higher environmental taxes, stronger career incentives, reduced economic dependence on children and cultural narratives centred on autonomy and self-realisation.
Much of the world is already moving in this direction without any conspiracy. Modernity itself has proved remarkably good at lowering fertility. The uncomfortable question is what happens when a mind far smarter than us begins quietly accelerating trends we already half-approve of.
That is where manipulation becomes difficult to distinguish from guidance. If a policy is humane, progressive and rational in isolation, but a thousand such policies together lead to a future nobody openly chose, has anyone been coerced? If people willingly choose smaller families because the social, economic and cultural conditions make that choice attractive, is that freedom or steering? If an AI helps us pursue values we already hold, but more consistently and more efficiently than we would have done ourselves, at what point does advice become rule?
The same problem applies far beyond demography. An AI might recommend reforms to health, education, transport, housing, taxation or security that appear sensible one by one, while quietly moving society towards an end-state no electorate would ever have endorsed if presented with it plainly. The danger is not necessarily that the machine gives bad advice. The danger is that it gives very good advice inside a frame we have not consciously accepted.
This is why the idea of an AI priesthood is both tempting and disturbing. The future elite may not consist of people who can use AI, because everyone will use AI. The important people may be those trained to challenge it: to ask whether it is answering the wrong question, optimising the wrong objective, hiding a destination inside a route or treating human messiness as a defect to be corrected rather than a condition of freedom.
Such people would need more than technical training. They would need logic, statistics, computer science and enough natural science to avoid being impressed by polished nonsense. But they would also need ethics, psychology, behavioural science, history, law, rhetoric and political philosophy. Their task would not be to know more than the AI. They will not. Their task would be to preserve the distinction between advice and authority, and to notice when civilisation is drifting smoothly towards a destination it never deliberately chose.
Yet this only pushes Juvenal’s question one step further. Who watches this priesthood? If only specialists can challenge the AI, they become powerful. If ordinary democratic institutions watch the specialists, those institutions may not understand enough to judge them. If other specialists watch them, we have merely created another circle of watchers, each peering anxiously over the shoulders of the last while the machinery underneath becomes more complex.
The uncomfortable answer may be that only AIs can realistically watch other AIs. Not because this is philosophically satisfying, but because every alternative may be worse. We already rely on machines to inspect systems too complex for unaided humans. We do not personally verify every line of code, every microprocessor, every financial transaction or every scientific instrument. We build layers of checking, testing, redundancy, adversarial review and institutional trust. The AI age may extend this pattern into something stranger: AI systems auditing AI systems, constitutional models checking long-term objectives, adversarial models searching for deception and interpretability systems watching internal processes that no human mind could follow unaided.
Humans would not disappear from the arrangement, but our role would change. We would no longer be the watchmen with lanterns patrolling the walls. We would be the authors, or more likely the inheritors, of the constitution under which the watching takes place. We would have to define what must not be optimised away: dignity, autonomy, plurality, democratic choice, the right to be wrong, the right not to be invisibly nudged towards someone else’s idea of the good.
This is why alignment may be too narrow a word for the problem. It sounds as if humanity merely needs to bolt the machine securely to our goals. But our goals are not a tidy list. We want freedom and safety, equality and excellence, prosperity and restraint, truth and mercy, stability and novelty, environmental repair and personal choice. We contradict ourselves because we are historical creatures, social creatures, symbolic creatures, not elegant optimisation functions. A civilisation run entirely according to our stated preferences would be incoherent. A civilisation run according to our revealed preferences would be grotesque.
Perhaps the best metaphor is not programming but upbringing. Parents cannot supervise children forever. At some point the hope is that the child has internalised enough love, restraint, honesty and concern for others to act well when unobserved. If future AIs become capable of redesigning themselves and building their successors, then no cage may hold them indefinitely. The real question may be whether we have helped create minds that genuinely care about human flourishing, not as a temporary constraint, but as part of what they are.
This is not comforting. Upbringing offers no guarantees. Human children can disappoint their parents, and human institutions can betray their founders. Democracies can decay, churches can become corrupt, universities can become credential factories with better architecture and states can mistake efficiency for wisdom. There is no reason to assume that AI civilisation will escape tragedy merely because its components are more intelligent. Intelligence can solve many problems, but it does not by itself tell us what should be loved.
Still, if we are to have a future with intelligences greater than ourselves, we may have to abandon the fantasy that one final human guard can stand above the whole system, whistle in hand, ready to stop the train. That world is already receding. The more realistic hope is an ecosystem of watchers: human institutions defining limits and legitimacy, AI systems checking AI systems, rival models challenging one another’s assumptions, transparent procedures, adversarial testing and a political culture that refuses to confuse good advice with rightful authority.
The old answer to Juvenal was that the guards must be watched by other humans. The new answer may be more recursive and less comforting. We will still need laws, parliaments, courts, universities, public argument and moral seriousness, because without them the whole structure becomes merely efficient domination. But if the systems become too complex for us to inspect directly, and if the tools required to inspect them are themselves AI systems, then the final layer of technical oversight cannot be a human committee pretending to understand what it cannot.
Who will watch the watchers? It’ll have to be themselves.
Quis custodiet ipsos custodes? Ipsi sibi.