Why Does AI Make Things Up? (And How to Spot It)

In October 2025, Deloitte agreed to partially refund the Australian government for an A$440,000 report. The report was reportedly "littered with apparent AI-generated errors," including a fabricated quote from a federal court judgment and references to academic research papers that don't exist. [1]

That's a big firm, paid a lot of money, with professionals whose job is to check this stuff. If it can slip past them, it can slip past the rest of us. So here's what's actually going on when an AI makes things up, and what you can do about it.

What is an AI hallucination?

It's when an AI gives you something that sounds right and isn't. OpenAI defines it as "plausible but false statements generated by language models." [2] The key word is plausible. It doesn't look like a mistake. It looks like every other answer: confident, tidy, well written.

It tends to show up in a few forms:

• A source or reference that doesn't exist, often with a real-sounding author and title.
• A quote nobody said.
• A date, number or statistic that's slightly or completely wrong.
• A feature of a product or tool that it doesn't actually have.

Why does AI make things up?

There are two reasons, and neither is "it's lying."

First, it's predicting, not looking things up. Think of the predictive text on your phone, which suggests the next word based on what usually comes next. A chatbot is a far more powerful version of that. It was trained to predict the next word across huge amounts of text, so it learned the patterns of language very well. It did not learn a neat database of true facts.

For common things, patterns are enough, and it gets them right. For obscure, specific facts, they aren't. OpenAI's own example is a pet's birthday. Facts like that, it says, "cannot be predicted from patterns alone." [2] There's nothing in the pattern to tell it the answer, so it produces something that fits the pattern of an answer.

Second, the way these models are tested rewards guessing. OpenAI's explanation is the multiple-choice exam. Leave a question blank and you get zero. Guess and you might get it right. If the scoreboard only counts correct answers, a model that always guesses beats one that sometimes says "I don't know." So models end up trained to guess confidently instead of admitting they're unsure. [2]

The good news is that OpenAI says models can learn to hold back when they're unsure, and that it's working on it. The less comforting news is that they also describe hallucinations as "a fundamental challenge for all large language models." [2] Newer models make fewer obvious blunders, which I'd argue makes the remaining ones harder to spot.

Does it really matter in practice?

It's already turning up in places where mistakes are expensive. Damien Charlotin, a researcher at HEC Paris, keeps a database of court filings with AI-fabricated citations. As of its 31 October 2026 update it listed 2,246 court decisions where generative AI produced hallucinated content, [3] up from about 200 in mid-2025. [4] Part of that rise is simply more people using AI and more courts looking, but it's a steep climb, and it includes lawyers who should know better.

How can you spot it?

The tells are not what you'd expect. It won't sound unsure, it won't hedge, and it won't change tone. If anything, the more specific the detail, the more convincing it feels. Names, numbers, quotes and citations are the riskiest parts, because they're exactly what a pattern-matcher is most likely to invent.

There's one more catch. The easiest hallucination to catch is the one in a subject you already know well. In a subject you don't, you have nothing to compare it against. That's a big reason I think learning the basics first matters so much, which I wrote about in The 10,000 Steps Problem.

What can you do about it?

You won't get rid of hallucinations, but you can catch most of the ones that matter.

1. Check the checkable parts first. Names, numbers, quotes, links and citations are quick to verify and are where the errors hide. If it names a study or a case, search for it.
2. Open the sources, don't just count them. A neat list of references isn't proof. Fabricated ones look just as real, so click through and confirm they exist and say what it claims.
3. Give it the material. Paste in the document or the facts and ask it to answer only from that, and to quote the line it used. It has far less room to invent when the answer is sitting right there.
4. Ask for evidence, and give it permission not to know. I use a line like "Use evidence before making a statement, and if you're not sure, say so." Claude's maker lists both ideas, permission to say "I don't know" and asking for a supporting quote for each claim, among its ways to cut hallucinations. It also says they reduce them, not eliminate them. [5] One catch: with a general question, the "evidence" can be invented too, so treat it as a prompt to check (point 2), not a replacement.
5. Ask again, in a fresh chat or a different tool. If you get a different answer, you've found a shaky spot. If the answers match, that's reassuring but not proof, because they can be wrong in the same way. Researchers at Oxford found that asking a model the same question several times and having a second model compare the answers picked out likely errors about 10% better than asking one model to double-check itself, though that covers only a small corner of the problem. [6]

And scale your effort to the stakes. Dinner ideas don't need a fact-check. Anything involving money, health, legal matters, or something with your name on it does.

Is it really different from asking a person?

Less than it sounds. People make things up too. Sometimes it's honest: "I swear it was X," and it was Y. Memory isn't a recording. Elizabeth Loftus's experiments found that even the wording of a question can change what witnesses later report. [7] Sometimes it isn't honest at all. The specialist you're talking to may want your business, and the advice that wins the sale isn't always the advice that's best for you. An AI isn't trying to win you over. It's guessing what an answer looks like, as above.

And the AI's mistakes are at least somewhat predictable. They cluster in names, numbers, quotes and citations, which is why the checklist above works. A pitch or a confident memory doesn't come with a tell like that.

So the vigilance you'd give a person advising you is about right for an AI. Take it seriously, check what matters, and check harder when the stakes are high or when the answer happens to suit whoever is giving it.

Where this leaves you

The confidence is in the writing, not in the facts, and the AI doesn't know the difference. So the checking is your job. Treat it like a very well-read colleague who occasionally makes something up without noticing, and never changes their tone when they do. You'd still work with them. You'd just check the parts that matter.

Sources

1. CTV News (AP), Deloitte to partially refund Australian government for report with apparent AI-generated errors, 7 October 2025.
2. OpenAI, Why language models hallucinate.
3. Damien Charlotin, AI Hallucination Cases database, last updated 31 October 2026 (2,246 cases). The database tracks legal decisions, not every filing that used AI, and grows as new cases are found.
4. HAQQ, AI Hallucination Cases: the 2,202-Case Sanctions Tracker, which reports Charlotin's database as having about 200 decisions in mid-2025. Secondary source.
5. Anthropic, Reduce hallucinations. It recommends allowing Claude to say "I don't know" and using quotes to support claims, and says the techniques "significantly reduce hallucinations, [but] they don't eliminate them entirely." This is guidance from one AI maker.
6. Scientific American, Can One Chatbot Catch Another's Lies?, on research including Jannik Kossen (University of Oxford). The page shows no publication date.
7. Macmillan Learning, Misinformation Effect, a textbook companion page summarising Elizabeth Loftus's research.