The one idea
Fabrication is not a bug that will be prompted away. It is a consequence of how these tools generate text, and it will still be there next year.
So the question is never "how do I stop it making things up?" It is:
Which parts of this output am I unable to judge by reading it, and what happens if those parts are wrong?
That question cuts your checking to a small fraction of the output, and stops the two failure modes people fall into — checking nothing, or being so anxious they verify a brainstorm.
Check the facts, not the writing. If you could tell it was wrong by reading it, it does not need checking.
Sort every output into ideas, transformations, and claims. Only claims need external verification, and only in proportion to what acts on them.
Verification cost should scale with consequence. Retrieval-grounded answers reduce fabrication but do not eliminate it — the generated sentence can still exceed what the retrieved source supports.
What needs checking and what does not
| Output type | Example | Check? |
|---|---|---|
| Ideas | Ten angles for a campaign, possible causes of a bug | No — you judge these by using them |
| Transformations | Summarise this, rewrite this, translate this | Check against the original, not the world |
| Explanations | How does an index fund work | Lightly, if it matters and it is unfamiliar |
| Specific facts | Numbers, names, dates, quotations, statistics | Always, at the source |
| Citations and references | Reports, cases, papers, sections, URLs | Always, by opening them |
| Regulated claims | Legal, medical, tax, financial, safety | Always, and usually with a professional |
Two lines in that table carry most of the risk.
Transformations get checked against the source, not the internet. When it summarises a document you supplied, the failure is not usually invention — it is omission, or a confident statement about something the document merely implied. Read the summary with the original open.
Citations get checked by opening them — not by recognising the journal or noticing the author is real, but by reading the sentence they are supposed to support.
How to spot a fabricated citation
Fabricated references have a texture once you know it.
- The shape is perfect and the specifics are round. Plausible author, plausible journal, a clean year. Real references are messier.
- It sits exactly where you needed support. You asked for evidence for a claim, and evidence appeared, precisely fitted to it.
- The title describes your question almost too neatly. Real papers have narrower, more awkward titles than the argument they get cited for.
- The link 404s, or leads somewhere real that says something else. The second is more common and more dangerous.
- You cannot find it by title in quotes. A real published document with an exact title is findable. If a quoted-title search returns nothing, that is close to conclusive.
The check that catches nearly all of it takes fifteen seconds: search the exact title in quotation marks. If nothing comes back, it does not exist. If something comes back, open it and confirm it says what was claimed — a real document cited for a claim it does not make is the harder and more common error.
You get five references for an essay. Three resolve. One does not exist — the authors are real, both publish in the field, and they never wrote this paper together. One exists but argues the opposite of what it was cited for.
If you had submitted it, the fabricated one would probably have been caught, and the misrepresented one would have been worse: it looks like you read a source and misdescribed it, which is a different accusation.
You ask for the notice period rules that apply to an employee. You get a confident answer with a section number. The correct use of that answer is as a search query, not as a conclusion: it tells you which Act and which concept to look up. You then read the provision yourself, or ask HR, and discover the rule depends on the state and on what the contract says — a qualification the confident answer dropped entirely.
A verification pass that takes four minutes
1 of 6Mark every specific claim. Read once with a highlighter mindset. Numbers, names, dates, quotations, citations, legal or medical statements. In a typical answer this is three to six items, not the whole thing.
When you have already used a wrong answer
You will, at some point. Recovery is a procedure, and doing it quickly costs far less than doing it well later.
Recovering from a wrong answer you have already sent
1 of 6Confirm it is actually wrong, at the source, before you say anything. A retraction based on a second unverified impression is worse than the original error.
Do not blame the tool in the correction. You used it; the output is yours. "The AI got it wrong" reads, correctly, as "I did not check", and says you may not check next time either.
Try this
You receive this from a colleague, produced with AI help:
"Under Section 25F of the Industrial Disputes Act, 1947, an employer must give one month's notice before retrenchment. A 2023 NITI Aayog study found this is followed in only 34% of cases."
Which parts would you verify, in what order, and how?
Your challenge
Level 3 · IndependentAsk an AI tool for five sources on a topic you know something about. Ask for titles, authors, and years.
Verify all five. For each, record one of: exists and supports the claim; exists but does not support the claim; does not exist.
You have succeeded when you have done all five and can describe what — if anything — distinguished the unreliable ones from the reliable ones in the output itself, before you checked.
Most people find the honest answer is "nothing". That is the finding. It means the only workable policy is checking every citation you intend to rely on, not the ones that feel shaky.
What people usually get wrong
- Asking the AI to verify itself. "Are you sure?" and "check this" generate more text by the same process. An apology is not evidence.
- Checking with a second AI tool. Two systems trained on overlapping text agreeing tells you very little. This feels like corroboration and is not.
- Accepting a citation because the source is real. Real publisher, real author, non-existent document is the standard shape.
- Verifying the easy parts. People check spelling and skip the number.
- Checking everything. Verifying a brainstorm wastes the budget you needed for the one figure going into the deck.
- Only checking for invention. Omitted qualifications cause more real damage than fabrications, and they are invisible unless you ask what conditions apply.
- Hiding a discovered error. The cost of a corrected mistake is small. The cost of one found later by someone else is not.
How someone experienced does it
Experienced users design the task so verification is cheap, rather than making verification heroic afterwards.
Concretely: they supply the source material instead of asking from memory. They ask for quotations with locations — "quote the exact sentence and give the page or section" — because a quotation with a location is checkable in seconds, while a paraphrase is not. They ask for the answer and the confidence separately: "which parts of this would you expect to be reliable, and which are you least certain about?" The self-report is not a measurement, but it does tend to point at the thin ice, and it is free.
The habit underneath all three: make the output falsifiable. An answer you can check in thirty seconds gets checked. An answer that would take an hour to check gets believed. So they push work into the first category by construction — and that, not diligence, is why they get caught out less often.
When not to use this
Full verification is wrong for low-stakes, self-evident output. Do not fact-check a list of possible titles for your presentation, or a rewrite of your own paragraph. You are the judge and the answer is in front of you.
It is also the wrong response to a systematically unreliable task. If you are repeatedly verifying and repeatedly finding errors, stop verifying and change the job — supply the documents, or use a different method. Verification is for catching occasional errors, not for filtering a stream of them.
Prove it
Take the last substantial thing you produced with AI help. Do the four-minute pass on it now: mark the claims, drop the ones that do not matter, verify the rest at source, and write one line per fact with where you found it.
Keep the list. It is the difference between "I think that's right" and "here is where it came from" — and that difference is the whole of professional credibility.
Keep learning this
Paste this into any AI assistant. It turns the assistant into a tutor that tests you instead of just answering you.
Act as an experienced practitioner who is good at teaching. I have just learned verifying AI output and checking for fabricated citations. Assume I am intelligent but relatively new to this — treat me as intermediate level. Work through this in order, and wait for my reply at each step: 1. Ask me 5 questions that test whether I actually understood verifying AI output and checking for fabricated citations. Do not reveal the answers yet. 2. After I answer, tell me which parts I got right, which I got wrong, and which I only half-understand. Explain only what I misunderstood — do not re-teach what I already know. 3. Give me one practical challenge based on something I could genuinely encounter at work or in daily life. Do not solve it for me. 4. Evaluate my solution the way an experienced person would judge it, including what a professional would have done differently. 5. Tell me what to learn next, and why that comes next. 6. Give me trustworthy sources for deeper study — prefer official documentation, primary research or standards bodies over blogs and videos. Rules for you: no buzzwords. No motivational filler. Say "I'm not certain" when you are not certain, and tell me which parts of your answer I should verify myself. Clearly separate facts from your recommendations and your opinions.
Become independent at this
Use this when you want a path from where you are to actually good, with checkpoints you can test yourself against.
I want to become independently capable at checking AI output before you rely on it — not permanently dependent on AI, tutorials or step-by-step guides. Design a progression for me with five stages: Beginner, Guided practice, Independent practice, Real-world application, Professional level. For each stage tell me: - what I must know - what I must be able to do without help - the mistakes people make at this stage - one practical challenge - one real project that would prove I reached this stage - one way I can test myself honestly Then tell me the signals that I am ready to move to the next stage, and the signals that I have skipped ahead too early. Keep the theory to the minimum I actually need. Focus on ability I can transfer to situations you and I have not discussed.