AI hallucinations and law firms: how to contain the risk

An AI hallucination is output that is fluent and confident but false: a case that does not exist, a real case quoted as saying something it does not, a wrong date or figure. In 2025 the High Court dealt with fake citations put before it and said lawyers must check AI research against authoritative sources. Containing the risk means checking every fact and authority before anyone relies on it.

Most articles on this topic stop at 'always check the output'. That is correct and not very useful, because it does not say what to check, how, or who. This piece goes through what the court actually said, where hallucinations turn up beyond case citations, and a routine a firm can adopt as it stands. It sits in our guide to AI for UK law firms. It is not legal advice.

What is an AI hallucination in legal work?

Language models generate text that is statistically likely to follow the prompt. They do not look facts up unless they are built to, and even then they can blend what they found with what they predict. The result can be:

• A case that does not exist, with a plausible name and neutral citation.
• A real citation attached to the wrong case.
• A real case described as deciding something it did not decide.
• A quotation that does not appear in the source it is attributed to.
• A statutory provision that has been amended, repealed or never existed in that form.
• A wrong date, amount, name or time limit in an otherwise accurate summary.

The Divisional Court described the problem in almost exactly those terms: freely available generative AI tools can produce coherent and plausible responses that turn out to be entirely incorrect, make confident assertions that are untrue, cite sources that do not exist, and purport to quote passages that do not appear in the source.

Which UK case dealt with fake AI citations?

The leading judgment is R (Ayinde) v London Borough of Haringey and Al-Haroun v Qatar National Bank QPSC [2025] EWHC 1383 (Admin), handed down on 6 June 2025 by the Divisional Court (the President of the King's Bench Division, Dame Victoria Sharp, and Mr Justice Johnson). The court heard two referrals together under its Hamid jurisdiction, which concerns the court's power to enforce the duties lawyers owe to it.

• In Ayinde, grounds for judicial review cited five cases that did not exist, including one presented as a Court of Appeal decision. When the other side asked for copies, none were produced. A wasted costs order was made at an earlier hearing, and the Divisional Court referred the lawyers involved to their regulators.
• In Al-Haroun, a schedule of forty-five citations put before the court included eighteen cases that did not exist. Of those that did exist, many did not contain the quotations attributed to them or did not support the propositions they were cited for. The solicitor accepted that he had relied on legal research carried out by his own client, without independently verifying it. One of the fictitious authorities was attributed to the very judge hearing the application.

The court decided not to begin contempt proceedings in either case, but made clear that placing false material before the court can lead to wasted costs, regulatory referral and, in serious cases, contempt or worse.

What did the court say lawyers must do?

Three points from the judgment matter most for a firm.

• Check against authoritative sources. Lawyers who use AI for research have a professional duty to check its accuracy against authoritative sources before using it to advise clients or before a court. The court named examples: the government's legislation database, the National Archives database of judgments, the official Law Reports and reputable legal publishers' databases.
• The duty applies to reliance on others. It rests on lawyers who use AI themselves and on those who rely on the work of others who have. The court compared it to relying on a trainee's work or an internet search.
• Leadership is on the hook. The court said practical and effective measures must now be taken by those with leadership responsibilities, naming managing partners and heads of chambers, and that future hearings of this kind can be expected to ask whether those responsibilities were fulfilled.

Al-Haroun is the one to remember. The false citations did not come from the solicitor's own use of AI at all. They came from material a client supplied, which the solicitor passed on to the court unchecked. The court's words were that a lawyer is "not entitled to rely on their lay client for the accuracy of citations". The duty to verify did not change.

Where else do AI hallucinations show up in a law firm?

Citations get the headlines because they reach judges. In day-to-day practice, the quieter hallucinations get less attention and are just as capable of harm, because they reach clients rather than judges, and clients are less likely to check.

• Summaries. A summary of a long email chain that states the wrong date for an agreed deadline, or leaves out the one sentence that changes the position.
• Client letters. A drafted update that fills a gap with a plausible fact that is not on the file.
• Call notes. A transcript summary that attributes something to the caller they did not say.
• Client-supplied material. As Al-Haroun shows, research or 'what the internet told me' from a client may itself have come from an AI tool, and arrives looking like fact.
• Front-door conversations. A chatbot or voice assistant that answers a question it should have declined, with a confident but wrong statement of the law.

The last one is the reason a client-facing system should be built to decline rather than answer. The SRA, authorising Garfield.Law in 2025, noted that its system cannot propose case law at all, which it called a high-risk area for language models. Removing a capability is more reliable than checking it. Our piece on what AI chatbots can safely say puts that into a say/never-say list.

How can a law firm prevent AI hallucinations?

You cannot stop a model generating them. You can stop them reaching anyone. We use a routine with four parts, which we call source, trace, sign and sample.

• Source: every authority is opened in an authoritative source by a person before it is used. Not searched for. Opened, and the relevant passage read. If it cannot be found, it does not go in.
• Trace: every factual statement in an AI draft that matters (a date, an amount, a name, a deadline, a quotation) is traced to the file or the source document. Ask tools to give references so tracing is quick; treat anything without one as unverified.
• Sign: a named person signs off the document, and their sign-off means they did the first two steps. If a trainee or paralegal prepared it with AI, the supervising fee-earner still checks the authorities, because the duty follows reliance.
• Sample: once a month, the partner responsible for AI picks a few AI-assisted documents at random and re-checks them. This catches drift, when checking gets lighter as people get used to a tool.

Put the routine in your AI policy as a single line, and in your supervision process as a habit. AI governance for law firms has a one-page policy that includes it.

A checking routine, worked through

A hypothetical example, invented for illustration. A trainee uses an approved AI tool to prepare a research note on a landlord and tenant point. The note cites four cases and one statutory provision. The supervising associate opens each case on a reputable database: three say what the note says; one exists but concerns a different point; none is fictitious. She opens the provision on the government's legislation site and finds it was amended after the date the tool seemed to assume. Two of five items needed correction. Checking took around forty minutes. The note that reaches the partner is accurate, and the file records who checked what. That forty minutes is the cost of using the tool. It is still worth using if the note would have taken longer to write from scratch, and it is not worth using if it would not.

Do AI tools built for lawyers still hallucinate?

Tools built for lawyers often connect the model to a database of real authorities, which reduces the risk of wholly invented cases. It does not remove the risk of real cases being mischaracterised or quoted inaccurately. The court's duty to check does not depend on which tool was used, and no vendor's assurance changes that. See what Harvey AI is for an example of how one major legal tool describes its research features.

What about agents that act on AI output?

A hallucination in a draft waits for a reader. A hallucination inside an agent that sends messages or updates records can be acted on before anyone reads it. That is the strongest argument for keeping agents' automatic actions few, reversible and free of legal effect. AI agents for law firms sets out how to draw that line.

How WasilX handles this

We designed the front door to decline rather than answer. Cam, our AI Engagement Coordinator, says it is AI, asks the firm's own qualifying questions, books the call back and sends the firm a summary. It reads no documents and uses no case law, so it has nothing to cite, and it does not give legal answers. The fee-earner reads the summary before calling, which is itself a check. For the confidentiality side, see AI and client confidentiality. If you want to see how it behaves when a caller pushes for advice, we are happy to show you.

Frequently asked questions

What happened in Ayinde v Haringey? Grounds for judicial review cited five cases that did not exist. The Divisional Court, hearing it with Al-Haroun in June 2025, did not begin contempt proceedings but referred the lawyers involved to their regulators and set out the duty to verify AI research.

Can a solicitor rely on research a client provides? The court in Al-Haroun treated a solicitor's reliance on his client's research, without verifying it, as a serious failure. The duty to check authorities applies whoever did the research.

Which sources count as authoritative for checking AI output? The court gave examples: the government's legislation database, the National Archives database of court judgments, the official Law Reports and the databases of reputable legal publishers.

Can AI hallucinations be eliminated? Not at the source. They can be kept from reaching clients or courts by checking every authority and material fact before use, and by building client-facing tools that decline to answer legal questions.

What are the consequences of citing a fake case? The judgment lists possibilities including wasted costs orders, referral to the regulator and, in serious cases, contempt proceedings or police referral.