The Limits of Artifical Intelligence in Adjudication: Analysing The Supreme Courts Zero-Tolerance Approach to Ai-Generated Fake Precedents

INTRODUCTION

Artificial Intelligence (AI) has revolutionized the legal profession by making legal research, document review, drafting, and case analysis quicker. Previously time consuming manual processes are now completed in minutes, boosting efficiency and access to legal information. The same technology, however, poses a serious threat to the administration of justice: that of false legal precedent being referenced as real precedent.

Generative AI is different from traditional legal databases, which generate predictions or responses by recognizing patterns in the training data. This can be useful for research assistance, but can also result in "hallucinations" by the AI system, that is, producing ostensibly correct case names, citations or legal propositions which are completely fictional. These outputs might sound like a real judicial authority and can be extremely harmful when taken for granted without being verified. Their use may affect judicial reasoning, erode the principle of precedent and erode public trust in the justice system.

In a ground breaking ruling that highlighted this emerging challenge, the Supreme Court in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. & Anr., 2026 INSC 668 addressed the issue of the responsible use of Artificial Intelligence in adjudications. Reliance on “falsified” or “non-existent” or “AI-generated” “precedents” is an attack on the integrity of the judicial process itself, the Court said. Apart from the orders of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT), it had adopted a zero-tolerance policy towards bogus legal authorities and had issued guidelines for the future use of AI in the courts.

It is important to note that the judgment does not ban Artificial Intelligence. Rather, it refuses substitution of the man's judgement. While recognizing the potential of AI to enhance legal research and judicial effectiveness, the Court highlighted that adjudication is a human endeavor that demands independent reasoning, accountability, and constitutional responsibility. AI can help judges and advocates, but can't replace judicial discretion. The Court thus emphasized that there should always be a “human in the loop” to check all the legal authorities and all conclusions made by technology.

There is another philosophical statement about the nature of decision making in the judgment as well. The Bench believes that the capacity for discrimination, for discernment between truth and falsehood, justice and injustice, grows with a continuous discipline of the mind and life. This process, the Court said, it is called “Saadhana,” and “a heavy reliance on AI might over time undermine the human traits that make up judgment.” The worry here is not only technological, it's constitutional: When we turn adjudication to the machine, the legitimacy of adjudication itself is called into question.

AI AS AN AID, NOT AN ADJUDICATOR

The Supreme Court discussed the wider implications of Artificial Intelligence in the legal profession before delving into the controversy. It accepted that AI has changed the nature of legal research, drafting and case analysis, with the ability to digest enormous amounts of information in just a few seconds. AI can be used to produce legal arguments and simulate analytical thinking, which is a skill some of the previous technology innovations, like e-filing, virtual hearings and digital databases, could not. The Court was thus led to believe that AI is different from the previous technological tools as it can affect the decision making process of the judiciary, not just to support it.

The Court did not have a dim view of technological advancement. Rather it acknowledged that AI can increase the efficiency of judges, save research time and, more generally, increase access to legal information. Its worry was a different one: the fact that legal professionals were increasingly using output generated by AI tools without exercising due diligence. As per the Bench, the technology issue is when conveniences take the place of critical thinking.

In order to make this distinction, the Court cited developments in the United Kingdom, where Garfield Law Limited, the first law firm to be authorised by the Solicitors Regulation Authority to offer regulated legal services, was allowed to do so. The Court acknowledged that this approval was accompanied by strong safeguards on AI hallucination, confidentiality, conflicts of interest and ongoing human oversight. Most importantly, no responsibility was placed on the software for the responsibility of everything generated, but instead, it would be on the qualified solicitor. The example had shown that a legal practice that is meaningful, with meaningful human accountability, is compatible with technological innovation.

The Court built on this example, making a clear demarcation between AI being an aid and AI being an adjudicator. AI can help judges and advocates find authorities, organize information and be more efficient. However, it cannot replace the exercise of independent legal reasoning. In contrast, the constitution provides accountability only for humans, and algorithms do not have the power to be held accountable.

The philosophy underlying this judgment is expressed in the Court's consideration of "Saadhana.The essence of this judgment is expressed in the Court's consideration of “Saadhana”. The Bench believes that adjudication is not a formula, but a craft that is learned and developed through ongoing study, practice and critical thinking. Distinguishing truth from falsehood, justice from injustice or constitutional values from technical legality takes time, and is achieved through constant mental exercise. Such capabilities cannot be duplicated simply with predictive software, no matter how complex and powerful.

The Court thus cautioned against the phenomenon of intellectual dependence on Artificial Intelligence. AI is undoubtedly making research easier, but over time, over-dependence on AI's analysis could thin the muscle of independent thinking. Research delegation can eventually turn into delegation of thought. This dependency jeopardizes not only individual cases but over time the legitimacy of courts, which are based on reasoned human judgment, not algorithmic prediction.

That's why the judgment repeatedly stresses the need for “a human in the loop.” All legal arguments created by AI need to be substantiated and validated on their own before they can be incorporated into judicial decision-making. Human oversight is thus no luxury but a constitutional necessity to ensure that technology does not replace judges but enhances their role.

Armed with this conceptual backdrop, the Court proceeded to the case that would give rise to a classic judgment on Artificial Intelligence (AI). It noted that the scientific reasons behind the occurrence of AI hallucinations rested with computer scientists and AI engineers. The judiciary was worried about the impact that bogus legal powers have on judicial proceedings, and how those bogus legal powers might affect judicial rulings.

The one that gets us most likely is the analogies the judgment draws between AI fabricated fake precedents and "the release of methyl isocyanate in the province of law and justice invisible, insidious and catastrophic by the time anyone notices. The analogy was meant to convey the Court's insight into the way fabricated authorities creep in through pleadings and judicial decisions without being noticed until they are discovered. They may by the time they are found have already fed into judicial reasoning or research on future judicial issues. The Court therefore did not consider AI hallucinations to be typical research errors, but rather a systemic danger to the integrity of the rule of law.
 

AN INSOLVENCY APPEAL BECAME AN AI LANDMARK

The appeal was filed by the debtor, Jammu and Kashmir Bank Limited, against a resolution plan that the Bank has submitted against Essel Infraprojects Limited (EIL) for insolvency proceedings under Section 7 of the Insolvency and Bankruptcy Code, 2016. The Borrower's default led to a Corporate Insolvency Resolution Process being initiated by the Bank against the Corporate Guarantor.

The appellant, suspended director of EIL, on the other hand argued that the liability of the guarantor had changed fundamentally due to its subsequent corporate restructuring by way of a demerger and amalgamation. It was also argued that the Bank's new sanction letter did not acknowledge the ongoing guarantee, implying that the guarantee was no longer in existence. The NCLT, however, rejected those applications, accepted the Section 7 application and imposed the statutory moratorium. The NCLAT was in no doubt and held that the guarantee deed was clear to state that the guarantor remained liable for some restructuring or amalgamation.

The Supreme Court would have reviewed the correctness of interpretation made by the tribunals in the guarantee deed if it were a normal case. Instead, the proceedings took an unexpected turn as the senior counsel for the appellant drew the attention of the NCLT to the fact that several authorities relied upon by the NCLT did not exist or had paragraphs which could not be traced to the relevant judgments. Some of the citations referred to real cases, but took quotations that were not in those cases; others were purely made up.

The Supreme Court did not take these allegations on its own but has verified each of the aforementioned authorities cited by the NCLT. The exercise uncovered a number of fabricated precedents and a number of wrong extracts that were taken into the tribunal's reasoning. The affidavit submitted by Jammu and Kashmir Bank, which explained that these authorities were not mentioned by counsel for it, was also important. The tribunal itself conducted research, and they seemed to have been put in place during that process, the Bank says. This changed the nature of the controversy from one about irresponsible advocacy to one becoming an institutional issue. The integrity of the adjudicatory process was thus at stake, if fabricated authorities could intrude into judicial reasoning with their own research, which they have carried out independently of the judge.

The Court thus did not wish to resolve the substantive insolvency matters. Rather, it believed that if the mind of judges was marred by the introduction of false or hallucinatory precedents, then the orders they issued would not be enforceable. Judgments of NCLT and NCLAT were quashed and the Section 7 application was remanded for fresh adjudication with NCLAT acting in a conscious manner refraining from taking any decision on the merits of the insolvency dispute.

HAVING A ZERO-TOLERANCE APPROACH TO FAKE AND HALLUCINATED PRECEDENTS

The most important thing the judgment said is that it's not enough to rely on arbitrarily made precedents: it's a violation of the rule of law if they don't exist or if they're generated by artificial intelligence. It is not the first time courts have found themselves with incorrect citation or wrongly argued cases, but in the present instance, it was entirely different on this occasion – the authorities were ones which were not even real, nor did they actually exist in law, but were fabricated to look as if they did.

The Court explained that it was not interested in the technical aspect of AI hallucinations. The sources of inaccuracy  either due to shortcomings in training data or due to the architecture of the large language models will be discussed by engineers and computer scientists. The concern of the judiciary is when the faked material intrudes into the legal process and is accepted as real legislation. Thereafter, it is no longer a technical problem but a constitutional problem  a legitimate ruling requires the use of real legal powers.

The Court's reasoning in this case is illustrated by the methyl isocyanate analogy. It noted that false precedents creep in "silently" in pleadings and orders, and their falsity would not be known until it is "discovered. They are like a poisonous gas, and people don't even know they are endangered until long after they have already affected the legal process. AI-generated authorities may appear like real decisions so can deceive advocates, judges and researchers if not confirmed otherwise.

The Court therefore concluded that fabricated authorities cannot be overlooked as a mere error by a researcher because they come from Artificial Intelligence. Most importantly, it stated that if even "an iota" of fake or hallucinated material is used in judicial reasoning, the conclusion of the judicial decision is not sustainable. The fault is not the conclusion but in the way the decision is arrived at. For the legitimacy of the judiciary, all stages of the judiciary must be based only on true legal sources.

PROFESSIONALISM AND ACCOUNTABILITY OF THE BAR AND THE BENCH

One of the striking things about the judgment is that it does not say it is entirely a technological issue. The Court noted that AI is “only a tool, it does not have a legal personality, it does not carry professional responsibility.” The responsibility remains with the advocates and judges who use its products. While AI can produce content, it is non-responsible and cannot guarantee its accuracy.

The Court reiterated that an advocate has a duty to represent not only the interests of their client, but also others. Each advocate has a professional duty to help the administration of justice by presenting the correct legal authorities to the court as an officer of the court. Produce a fake, non-existent or AI-generated precedent without independent corroboration is a professional misconduct. The Court did not accept this argument, suggesting that relying on AI cannot justify such shortcomings. Regardless of how complex the output of software can be, responsibility for it cannot be delegated.

Judges are equally bound and have a responsibility to impose such limitations. Counsel's role in legal research is typically an adjunct to the role of the court, but the court is ultimately responsible for judicial reasoning. Each precedent that is referred to in a judicial order must thus be checked against the true law for it to be part of the judicial reasoning. Mechanical acceptance of AI-generated content is not aligned with constitutional rights and responsibilities to decide cases without bias and with independence.

The Court's repeated focus on having a ‘human in the loop' exemplifies this shared responsibility. Human oversight is not just a technology safeguard, it's a constitutional necessity. Artificial Intelligence can organise information, identify potential authorities that might be relevant, and help draft, but the ultimate determination of facts, law and precedent is the responsibility of human decision makers. Algorithms should not be given the power to determine judicial discretion, ethical accountability and constitutional responsibility.
 

ADVICE TO THE BAR COUNCIL OF INDIA

The Supreme Court issued institutional directions to complement the judgment in response to the increasing use of AI. It instructed the Bar Council of India (BCI) to form an expert committee to look into the growing application of Artificial Intelligence (AI) in the legal profession and the issue of fake or hallucinatory placing of the authorities in front of courts.

The Committee's recommendations will include guidelines for the ethical use of AI, measures to prevent hallucinations of legal authorities, and appropriate disciplinary measures when advocates are not able to verify the material produced by AI. These directions recognise that regulation alone is not the right way to achieve technological innovation. Professional institutions need to set ethical guidelines that can keep up with the technological advancements as well.

Notably, the Court acknowledged that there is no need to replace the necessity of independent verification with improvements in AI technology. Even with high level intelligence systems, they can still give out wrong or made up products. Hence, the duty to verify legal authorities exists outside of the future development of technology. Ethical responsibility will never change irrespective of technology.

CRITICAL ANALYSIS

The judgment is one of the earliest in-depth judicial rulings on Artificial Intelligence in India and will likely serve as the blueprint for future decisions on AI-powered adjudication. Its most powerful appeal is that it recognizes that a fabricated precedent not only risks the accuracy of judicial decisions, but the legitimacy of the judicial process. It is not just about the proper course of procedure, it is about the constitutional obligation to be authentic, thus strengthening confidence in the administration of justice.

The Court's approach to technology is also very balanced. It does not view AI as a threat, but rather as a tool that can enhance legal research and judicial efficiency, and emphasize that technology should not supplant independent thought. This prevents the "over-optimism" and "over-rejection" of technology.

The judgment does, however, raise some important questions for development in the future. The "an iota" principle takes an uncompromising view that means that even if a judge uses the bare minimum of real facts, the opinion is nevertheless considered null because it relies on "fabricated material. This fosters the integrity of the institutions involved, but in the future, it may be necessary to examine the scope of the principle when the claim of an artificial authority proves to be false and yet is unevaluated in the final decision. Likewise, the Bar Council of India will need to establish objective benchmarks to differentiate between the intentional use of AI and unintentional reliance, even when the result is reasonable.

CONCLUSION

In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. & Anr., the landmark judgment in Indian AI law, the court's ruling will have a profound impact on how AI is perceived in the future. The Supreme Court has made it clear that although AI can enhance legal research and judicial productivity, it is not a substitute for human judgment, which is the essence of adjudication. For the judge, the legitimacy depends upon the verification of legal authorities, independent reasoning and constitutional accountability.

The Court has thereby demonstrated that authenticity without which there would be no rule of law, has taken a zero-tolerance approach to fake, non-existent and hallucinated precedents. It has been emphasising that even an "iota" of fabricated information can taint the adjudicatory process, highlighting the need to maintain public trust in judicial institutions.

Rather than it being a mundane insolvency matter, Pooja Ramesh Singh lays out a detailed roadmap for the judicious deployment of Artificial Intelligence (AI) in Indian courts. The thrust of the message is that technology can help the administration of justice, but human judges and advocates should always be the ones who are responsible for the administration of justice.