When AI enters courtroom, justice must remain human

The Supreme Court’s decision yesterday to set aside orders passed by NCLT and NCLAT on the basis of fake AI-generated citations is a timely warning. It is not a rejection of artificial intelligence. It is a reminder that in matters of law, AI can assist the mind, but it cannot replace judgment. A court system may use machines to become faster, but it cannot allow machines to make it careless.

The problem is not that lawyers used AI. The problem is that they used it without verification. Fake judgments, hallucinated precedents and non-existent citations are not minor technical errors. In law, a false citation can distort reasoning, mislead a judge, burden the opposite party and damage public confidence. A hallucinated case is not merely wrong information; it is a counterfeit legal authority.

India’s judiciary is overburdened. Millions of pending cases, repeated adjournments, bulky records and uneven access to legal assistance make a strong case for technology. AI can help in transcription, translation, cause-list management, case clustering, legal research, summarising pleadings, detecting contradictions, scheduling hearings, flagging limitation issues and making court records searchable. These are not futuristic ideas. They are practical tools for a system that needs speed without sacrificing fairness.

But the courtroom is not a call centre, and justice is not a productivity metric. The judicial function rests on facts, law, reason and conscience. AI may retrieve, organise and summarise. It must not silently invent, embellish or substitute. The human element must remain in the driving seat, as the Supreme Court rightly emphasised.

The first principle should be mandatory human verification. No lawyer, litigant or court officer should be permitted to rely on an AI-generated case, quotation, statute, circular or order unless it has been checked against an authentic legal database or official source. Every citation produced with AI assistance should carry a verification trail: case number, court, date, neutral citation where available, and source. “AI gave it to me” cannot be a defence. It should be treated as professional negligence.

Second, courts should insist on disclosure. If AI has been used materially in drafting pleadings, written submissions, case compilations or summaries, the filing should say so. This does not mean stigmatising AI use. On the contrary, disclosure normalises responsible use. It tells the court that the lawyer understands the tool and has taken responsibility for the output. Secret use of AI creates risk; disclosed and verified use creates trust.

Third, plagiarism must be addressed seriously. AI makes copying easier, but it also makes unattributed borrowing harder to detect. Legal drafting often draws on precedents, model clauses and earlier pleadings, but there is a difference between legitimate reliance and intellectual dishonesty. Where AI is used to generate arguments, the lawyer must still ensure originality, attribution and accuracy. Courts and bar councils should develop norms for AI-assisted drafting: quote only with source, paraphrase only after understanding, and never pass off another author’s reasoning as one’s own.

Fourth, official legal AI platforms should be developed for the judiciary and the Bar. Lawyers are currently using open, general-purpose tools that may not be trained on complete Indian law, may hallucinate citations, and may not protect confidentiality. India needs secure, court-approved AI systems linked to authenticated databases such as Supreme Court, High Court and tribunal records. Such platforms should not “imagine” law; they should retrieve law. Where the database does not contain an authority, the system must say so clearly.

Fifth, AI tools must be designed with guardrails. Any judicial AI system should have source-linking, audit logs, confidence indicators, version history and strict limits on generative output. A legal research tool should not produce a fabricated judgment because a user has framed a persuasive prompt. It should refuse unsupported answers. In law, silence is better than fiction.

Sixth, responsibility must be fixed. If an advocate files fake AI-generated material, the advocate remains accountable. If a junior drafts it, the senior remains accountable. If a law firm uses AI, the firm must have internal protocols. If a court registry uses AI to summarise records, the authorised officer must verify it. Accountability cannot be outsourced to an algorithm.

Seventh, judges need training, not just software. AI literacy should become part of judicial academies and bar training. Judges must know what AI can do, what it cannot do, and where its risks lie. A judge need not become a computer scientist, but must understand hallucination, bias, data limitations, deepfakes, prompt manipulation and confidentiality risks. The best defence against bad technology is not fear; it is informed supervision.

Finally, AI should be used to expand access to justice. Translation into Indian languages, simplified summaries for litigants, automated status updates, assistance for persons with disabilities, and easier access to legal aid can transform the citizen’s experience of courts. This is where AI’s greatest promise lies: not in replacing judges or lawyers, but in reducing the distance between law and the ordinary citizen.

The Supreme Court’s warning should therefore not lead to a retreat from AI. It should lead to a mature framework for its use. India’s judiciary should adopt AI with ambition, but also with constitutional caution. Efficiency is desirable; integrity is non-negotiable.

AI can help courts move faster. But justice must still move with reason, responsibility and truth. The machine may assist the law. It must never be allowed to counterfeit it.

The writer is a former Secretary to Government of India; Views presented are personal.