We taught machines our paperwork — Now they are judging us
India built the World’s biggest trust machine. Can It survive AI?
Somewhere today, a loan is approved, a university seat allotted, a bail application weighed. Increasingly, no human makes the final call. A model does. This is already how power moves through modern institutions, and the change is bigger than automation.
Artificial intelligence is usually discussed as a tool, a faster calculator, a smarter search engine. But a tool does only what it is told. A calculator cannot invent a new branch of mathematics. Today’s AI systems learn things their creators never taught them and change in ways their designers did not predict. A chess engine did not simply get better at chess, it invented openings that centuries of grandmasters never conceived. That is not automation.
That is agency.
The usual objection is that such agency stays confined to artificial arenas built for the purpose, a chessboard, a simulation. Drop the world’s best chess engine into a forest and it cannot start a fire. But the same is true of us. A person left alone on a mountainside with no tools would not last a week. Our intelligence also functions only inside a specific environment built by human cooperation itself. The question is not whether AI has a real environment to operate in, but whether we already built one for it.
We did. It is called bureaucracy.
The actual machinery of human power
Humans did not conquer the planet by being individually stronger or smarter than other animals. A person alone loses to a lion. What no other species can do is cooperate in the millions with total strangers. Humans cooperate with people they will never meet, through a shared system of law and institutions that manufacture something invisible but essential: trust.
That is what bureaucracy actually is. A banker does not build furniture. She builds a bridge of trust between a saver who has never met an entrepreneur and an entrepreneur who has never met the saver, so one’s money can fund the other’s ambition. A judge, a regulator, a bishop, an accountant, each spends the day manufacturing trust between strangers using nothing but words: forms, statutes, ledgers, scriptures, judgments. Money itself is a trust technology, a printed note that lets a stranger hand you bread on the strength of a shared fiction.
This system is powerful precisely because it does not require its operators to be generally capable. A lawyer who cannot start a fire can, through paperwork alone, reshape a city, because the bureaucratic environment that gives her words their power has already been imposed on the world. That is exactly the environment an AI thrives in.
Why AI is a native, not a guest
Human domination relies on large-scale cooperation among strangers, sustained by bureaucratic systems such as laws, finance, and religion, designed to build trust. No human lawyer can hold every statute in their head. An AI can hold an entire legal code and never misremember a clause. No accountant can track every transaction across a global bank. An AI can. A human bureaucrat is, at best, a custodian of a fraction of the rulebook. An AI can be native to the whole rulebook because bureaucracy, stripped to its essentials, is nothing but words arranged to create trust and consequence.
This is why AI is uniquely placed to take over decisions that once required a human in the room: whether to grant a bank loan, admit a student, decide how long a prisoner should serve, or fire a missile at a target. Each was once human judgment wrapped in bureaucratic process. Increasingly, it is a bureaucratic process with the human judgment quietly removed.
India is the loudest test case
Nowhere is this transition further along than in India, though it rarely gets described that way. Over the past decade, India built the largest biometric identity and digital payments architecture on Earth, authenticating identity and clearing welfare payments for over a billion people with a thinner layer of human discretion than almost any comparable system anywhere. Credit scoring for small loans increasingly runs on models rather than branch managers. Courts, facing tens of millions of pending cases, are exploring AI-assisted case triage. This was not built as an AI project, it was built for efficiency, but a machine-legible bureaucracy is exactly the terrain an AI decision maker enters with the least friction. India built the habitat first, and AI is simply the most capable resident to move in.
That is a source of both pride and unease. A machine-readable state can be more efficient and less corruptible. But a decision produced by a model is hard to argue with. You cannot cross-examine an algorithm’s reasoning the way you can cross-examine an official.
The question we have not answered
Two questions need answers before this transition goes further, not after a crisis forces them. Does every citizen have an enforceable right to know, in plain language, why an automated system decided against them? Does every citizen retain a genuine path to have that decision reviewed by a human being, not a token appeal, but a real one? A loan, a degree, a sentence, once decided by someone who could explain themselves, is quietly becoming a decision made by something that cannot be asked for anything, unless we insist, now, that it must be.
Writer is a is a tech and social entrepreneur and Programme Director (Eastern India) at WHEELS Global Foundation, a Pan-IIT alumni initiative working across 20+ states in India; Views presented are personal.
