What Are AI Hallucinations, and How Do You Spot Them?

In April 2026, the law firm Sullivan & Cromwell, one of the most prestigious firms in the United States, with a client list that has included presidents, central banks, and the largest companies on the planet, filed an emergency motion in a federal bankruptcy court in New York. The motion cited legal authorities the way every court filing does: with case names, quoted opinions, and statute references. When opposing counsel at Boies Schiller Flexner went to check those citations, many of them did not exist. At least 28 were erroneous. Some included quotations attributed to the court that had never been said. The firm's founder of its restructuring group apologized to the judge. The errors, he explained, had been generated by an artificial-intelligence tool, and the firm's own AI-use policies had not been followed.

If Sullivan & Cromwell can be tripped up by AI hallucinations, so can your bank, your doctor, your lawyer, your accountant, your child's teacher, and you. This guide explains what AI hallucinations are, why they happen, how to spot them, and how to verify AI output before you make a decision based on it.

What is an AI hallucination?

An AI hallucination is content that a language model generates which sounds factually correct but is false. The term covers anything the model makes up while appearing confident: invented case citations, fake statistics, fabricated quotations, imaginary source URLs, non-existent historical events, incorrect dates, misattributed authors, phantom drug interactions, fictional court rulings, and more.

The word hallucination is not quite accurate in a technical sense. A model isn't "seeing things that aren't there." It is producing a plausible continuation of a prompt based on patterns in its training data. When a real answer is not retrievable, the model does not stop. It continues. The output looks the same either way.

This matters because the appearance of authority, formal sentence structure, correct formatting, confident tone, technical vocabulary, is exactly what a language model produces by default. An invented citation in a legal brief reads like a real one. An invented statistic in a health article reads like a real one. An invented quote from a public figure reads like a real one. The surface text contains no signal that says, I am not real. Verification is the only check.

Why do AI systems hallucinate?

Three overlapping reasons.

First, language models are trained to produce fluent text, not to be truthful. They are optimized to predict the next token given the context. A fluent lie and a fluent truth are equally likely to satisfy that objective. Most AI systems layer additional training and tooling on top to reduce hallucination rates, but those layers are imperfect.

Second, models do not know what they do not know. A human expert who is asked a question outside their knowledge will often say, "I'm not sure." Language models have no internal self-audit that reliably triggers uncertainty. They will produce an answer with the same tone whether the answer is known, approximated, or invented.

Third, training data is sparse in many specific domains. A model may have seen millions of generic contract clauses and only a handful of citations from a specific state supreme court's recent rulings. When prompted for the latter, it is likely to interpolate, filling in what seems plausible using the patterns it has seen in the former. The output looks correct to a non-expert.

The five most common types of AI hallucination

1. Fabricated citations and sources

The most common hallucination in research, legal, medical, and academic contexts. The model produces a real-looking citation, Smith v. Jones, 847 F.3d 1234 (9th Cir. 2021) or Henderson et al., Journal of Clinical Oncology, 2019, that does not exist. The format is exactly right because the model has been trained on thousands of real citations.

Recent high-profile example: The Sullivan & Cromwell motion in the Prince Global Holdings bankruptcy, filed April 9, 2026, contained at least 28 such fabricated citations, including quotations attributed to the court that had no basis in any real opinion. Researcher Damien Charlotin has catalogued more than 330 known AI hallucination cases in court filings worldwide, and that figure is the tracked count, not the actual one. For the full case, see our coverage: Sullivan & Cromwell Apologized for 28 AI-Fabricated Court Citations.

2. Invented statistics

A model asked "how common is X" will sometimes return a number that has no origin. The number usually feels right, it lands in a plausible range, but the source citation, if offered, either does not exist or does not say what the model says it says. This is especially common in health, economics, consumer-behavior, and public-policy answers.

3. Fabricated quotations

A particularly insidious class. The model produces a quote attributed to a real public figure on a topic plausible for them to have spoken about. In some cases, the model invents the entire quote. In others, it stitches together fragments from multiple real sources into a new sentence that was never said. Named attribution makes the content feel authoritative even when the underlying quotation is false.

4. Wrong dates, versions, and events

Especially common with topics that have moved in the last year or two. A model may report a law as passed when it is still pending, or describe an executive who left a company as still holding the role. It may place a conflict's start date a year off or assign an award to the wrong recipient.

5. Non-existent URLs, tools, and products

Ask a model for a recommended tool, library, or resource and it will sometimes recommend something that does not exist, returning a plausible URL, a plausible product name, and a plausible feature list. This is a particular risk in coding, where hallucinated package names can be typo-squatted by attackers and used to deliver malware.

Real-world examples of AI hallucination damage

Legal: The Sullivan & Cromwell case is only the most recent example. In 2023, a federal judge in the Southern District of New York sanctioned lawyers who filed a brief with six fake case citations generated by ChatGPT. In the years since, courts in at least two dozen U.S. jurisdictions and multiple other countries have issued similar sanctions. The Damien Charlotin hallucination-cases database tracks the aggregate count, which has grown into the hundreds.

Healthcare: Health systems experimenting with AI-assisted clinical note summarization have reported hallucinated medication doses, invented allergies, and procedures that never occurred appearing in AI-drafted summaries. Most systems now require human physician sign-off on any AI-generated note for this reason.

News media: Several major news outlets that experimented with AI-generated summaries of their own reporting found the summaries introduced factual errors, misattributed quotes, flipped numbers, and in some cases reversed conclusions, at rates high enough to discontinue the experiments.

Consumer fraud: Scammers now use hallucination-prone AI systems to generate counterfeit "official documents" such as fake subpoenas, fake IRS notices, fake court orders, and fake police emails that cite invented statute sections and invented case numbers. Formality is no longer evidence of authenticity. Our guide on signs of an impersonation scam covers the patterns to watch for.

How to spot an AI hallucination in real time

No single check catches every hallucination, but the following habits catch most of them.

1. Check any specific citation or source before you rely on it. Names, case numbers, DOI references, statute sections, and URL paths should all verify in an external search. If the citation does not resolve, treat the surrounding claim as unreliable.

2. Watch for unusually round or unusually specific numbers. Hallucinated statistics often have a tell: they are either suspiciously round (65%, 50%, $1 billion) or suspiciously precise in a way that does not match how the underlying figure would actually be measured.

3. Notice when a citation format is correct but the details feel off. Real case names, author lists, and journal titles have patterns. Real citations reuse consistent jurisdictional or style conventions. Hallucinations often get the format exactly right and the underlying entity subtly wrong.

4. Be most skeptical on recent events and niche topics. Hallucination rates are higher on anything that happened in the last year or two and on anything that is narrow enough to have minimal training data.

5. Ask the model the same question twice with different phrasing. Genuine answers tend to converge. Hallucinations often diverge, because the model is interpolating fresh each time.

6. If something feels like it would be a bigger story than you've heard of, it usually is not real. A hallucinated Supreme Court ruling that would overturn a major precedent, a hallucinated medical finding that would have made global news, a hallucinated quote from a president that nobody else seems to be covering, these are flags.

How to verify AI output, step by step

The single most important habit you can build around AI is what Sullivan & Cromwell's internal policy, now very publicly tested, directs its lawyers to do: trust nothing and verify everything. Below is a practical, non-legal version.

For citations and sources

• Paste the citation into Google Scholar, Google, or the source database directly
• If the citation claims to be a court case, search CourtListener, Justia, or the relevant court's docket
• If the citation claims to be a journal article, search the journal's own archive
• If nothing resolves, do not use the citation

For statistics

• Search the exact figure plus "site:[primary-source-domain]"
• Government figures should resolve at agency websites (ftc.gov, fbi.gov, cdc.gov, census.gov)
• Academic figures should resolve at a peer-reviewed source you can open
• If the model offers a "recent study" without a URL, request the URL; if the URL does not resolve, move on

For named quotes

• Search the exact quoted sentence in Google in quotation marks
• A real quote will typically resolve at multiple outlets
• If the only "hit" is the AI conversation itself or an AI-generated derivative, it is very likely fabricated

For medical, legal, and financial advice

• Cross-check against an official primary source: the CDC, the FDA, the IRS, the SEC, the text of the law, the drug label, the clinical guideline
• Use the AI's answer as a starting point for what to look up, not as the final answer

For anything safety-relevant

• Bring a human expert into the loop
• Never act on an AI's answer alone when the cost of being wrong is high

Tools that help

Several categories of tool reduce hallucination risk meaningfully.

Retrieval-augmented tools, AI products that are configured to cite their sources and show you the underlying documents, are generally more reliable than unrestricted chat interfaces. They still hallucinate, but the citations are usually verifiable.

Fact-check browsers and extensions scan text for known claims and flag mismatches with reliable databases.

Plagiarism and AI-content detectors like GPTZero, Copyleaks, and Originality.ai can help you identify when content you've received was AI-generated in the first place, which is the upstream signal that hallucinations might be present.

AuthentiLens scans suspicious content (text, image, audio, video, social-profile, and website) for AI-generation signals as well as scam and impersonation patterns. If someone sends you a "court notice," an "FTC warning," a "medical report," or a "family emergency" message and you are unsure whether it is real, paste it into AuthentiLens and we'll flag the AI signals and the manipulation patterns in seconds. Run your scan now.

How to build an "AI hygiene" habit

Three principles you can apply every day.

Assume AI output is a first draft. Use it as a starting point. Edit, verify, and add human judgment before you ship anything: an email, a report, a decision.

Maintain a personal list of "things I will never rely on AI alone for." Medical advice. Legal advice. Financial decisions that exceed some threshold. Anything safety-related. Anything that cites a specific authority you have not personally verified.

Treat formality and fluency as the absence of evidence, not the presence of it. A well-formatted bill, a well-written email, a confident quote, these used to be weak signals of authenticity. They are not anymore. Only primary sources and independent verification carry that weight.

Sources

• Damien Charlotin, AI Hallucination Cases Database
• Bloomberg, Top Law Firm Apologizes to Bankruptcy Judge for AI Hallucination
• ABA Journal, Elite Wall Street law firm apologizes for error-laden motion created by AI