The one idea
Recognition is no longer verification.
For your whole life, hearing someone's voice or seeing their face has been proof of who they are. That assumption is now unsafe, and it is unsafe precisely because it is so deeply automatic that you cannot switch it off by deciding to.
You cannot solve this by looking harder. Detection advice — count the fingers, watch for unnatural blinking, listen for a flat tone — describes the artefacts of tools that are already a generation old. Every visible flaw is a bug someone is being paid to fix, and the advice degrades with each release.
So the defence cannot be perceptual. It has to be procedural: a way of confirming identity that does not depend on how convincing the voice or the face is.
Recommendation Agree a verification method with your family now, while nothing is happening.
A shared word, or a rule that any request for money is confirmed by hanging up
and calling back on the number already saved in the phone. Something the
person on the call cannot supply and cannot talk you out of.
A voice or a face on a call is no longer proof of who it is. Hang up and call back on the number you already have.
Verify identity out of band before acting on any instruction involving money, credentials or access — regardless of how the instruction arrived.
Synthetic media removes biometric familiarity as an authentication factor. Authentication must move to a channel or shared secret the attacker does not control.
What is actually possible now
Being specific matters, because vague warnings produce either panic or dismissal.
Voice. Convincing voice cloning from a short sample is widely available and cheap. A social media video, a voice note, a webinar recording or a few seconds of a phone call is enough source material. This is the most mature of the three and the one being used most against ordinary families.
Video. Face replacement in a live call is possible and has been used in targeted corporate fraud. Quality varies, and it is harder than voice, but "harder" is not "impossible" and the gap keeps closing.
Images. Photorealistic images of people who do not exist, or of real people in situations that never occurred, are trivially produced. Fake endorsements — a recognisable public figure appearing to promote an investment platform — are now a standard shape of fraud, and they work because the face carries trust the platform never earned.
Fact CISA and partner agencies have published guidance for organisations on
synthetic media threats, which recommends verification procedures rather than
visual detection as the primary defence.
Why "get them on video" is weakening advice
The standard advice used to be: if you are unsure, ask for a video call. A scammer with a text channel could not produce a face.
That defence is eroding, and it is worth understanding why so you know what still holds. It rests on the assumption that the attacker cannot generate live video convincingly. That assumption is no longer safe against a prepared, targeted attacker — and it has already failed in documented corporate cases.
The version that still holds is callback on a known number. Not because a phone line is inherently trustworthy, but because you chose the channel. The attacker controls what reaches you; they do not control a number that was already in your contacts before any of this started.
That is the durable principle underneath all of this and it is the same one from the scams lesson: verify through a route the other party did not supply.
A grandparent gets a call from a grandchild in distress, and is told not to tell the parents because they will be angry.
The secrecy instruction is the strongest signal in the whole call, stronger than anything about the voice. Real emergencies involving family do not require you to conceal them from the rest of the family. Secrecy is there because one phone call to anyone else ends the scam.
A finance team receives a video call from someone appearing to be a senior executive, authorising an urgent transfer, with the usual reasons why the normal process cannot be followed this once.
The control that survives is procedural and boring: payments above a threshold require confirmation through an established channel, by a second named person, with no exceptions for seniority or urgency. A process that can be overridden by someone sufficiently senior and sufficiently urgent is not a control — it is a suggestion, and urgency plus seniority is exactly the combination the attack supplies.
Setting up a verification method with your family
1 of 6Have the conversation now, not during an incident. Ten minutes at a meal. Explain the voice-cloning call in one sentence and that it works because the voice is genuinely convincing — not because the person who falls for it was careless.
Try this
A call comes from a number you do not recognise. It is your sibling's voice, upset, saying their phone is broken and they are borrowing someone else's, and they need money now.
What do you do, and what do you specifically not do?
Your challenge
Level 3 · IndependentDo the family conversation this week. Not eventually.
You have succeeded when every adult in your family — including the ones least comfortable with technology, who are the ones most likely to be called — can state the shared phrase and the callback rule from memory.
Then do the second half: search for your own name and see what audio and video of you is publicly available. Every recording of your voice is potential source material for a call to someone who loves you. You do not have to delete anything. But you should know what exists, and the people who might receive that call should know the rule.
What people usually get wrong
- Relying on detection tips. Finger counts, blink rates and flat intonation describe last year's tools. The flaws get fixed.
- Treating a video call as proof. It was strong evidence and is now weaker evidence.
- Improvising security questions on the suspect call. You reveal what you count as proof, and personal facts are often already known.
- Accepting a reason not to call back. "No time", "phone is dying", "do not tell anyone" — all scripted, all designed to prevent the one action that ends it.
- Assuming an endorsement is real because the face is. A recognisable person in an advert is not consent, and often not that person at all.
- Believing a video because it confirms what you already think. The most effective fakes are the ones you want to be true.
- Setting up the family rule and never practising it. An unpractised rule is not recalled under stress.
How someone experienced does it
People who think about this well apply the scepticism in both directions, and the second direction is the one most people miss.
The first direction is not believing what you see. The second is understanding that synthetic media also devalues genuine evidence. Once anything can be faked, anyone caught on a real recording can claim it was fabricated. This is sometimes called the liar's dividend, and it may end up being the more consequential effect. It means provenance — where a recording came from, who published it, whether the original source stands behind it — matters more than what the recording appears to show. That is the same question as in "Who is telling you this?", applied to media rather than text.
They also treat their own voice and face as material that circulates. Not with paranoia, but with the same awareness they would apply to a document — public video, voice notes forwarded in groups, and recorded calls are all potential source material.
And professionally, they design processes that assume identity can be faked. Any process where a sufficiently senior and sufficiently urgent person can bypass the check has no control at all. The controls that survive are the ones with no exception clause, because urgency plus authority is exactly what the attacker brings.
When not to use this
Do not let this collapse into believing nothing. Reflexive dismissal of all media is its own failure mode, and it is the one bad actors benefit from most — it makes real evidence deniable.
The workable position is not "everything is fake" but "recognition is not verification". Ask where a recording came from and who is standing behind it. For a call, verify the channel. For a published video, check whether the source that would know has published it too. That is a normal sourcing question, not paranoia, and it scales sensibly with what is at stake.
Why detection tools are not the answer for individuals
It is reasonable to assume software will detect fakes for us. It is worth understanding why that is not a plan you can rely on personally.
Detection and generation are locked together. Any reliable detector becomes a training signal that improves the next generation of the thing it detects, so detection tends to lag rather than lead.
Detection also tends to be probabilistic and to degrade in exactly the conditions you meet in practice — compressed audio on a phone call, a re-encoded forwarded video, a low-light clip. A confidence score on a compressed voice call is not something to make a payment decision on.
And the timing does not work. The decision you face is whether to send money in the next five minutes, to a person who is preventing you from checking. Even a perfect detector you had to upload something to would arrive too late.
Which is why the recommendation from official guidance is procedural. Verification through an independent channel does not care how good the fake is, and it does not stop working when the technology improves. That property is what makes it worth building now.
Prove it
Two things, both finished this week.
One: your family has a shared phrase and a callback rule, and has said both out loud at least once.
Two: you have told the most vulnerable person you know — the one who would be called — the single sentence that matters. If someone calls sounding like me and asking for money, hang up and call me back on my normal number, and I will never be upset that you checked.
Keep learning this
Paste this into any AI assistant. It turns the assistant into a tutor that tests you instead of just answering you.
Act as an experienced practitioner who is good at teaching. I have just learned AI-generated voice and video used in fraud, and how to verify identity. Assume I am intelligent but relatively new to this — treat me as intermediate level. Work through this in order, and wait for my reply at each step: 1. Ask me 5 questions that test whether I actually understood AI-generated voice and video used in fraud, and how to verify identity. Do not reveal the answers yet. 2. After I answer, tell me which parts I got right, which I got wrong, and which I only half-understand. Explain only what I misunderstood — do not re-teach what I already know. 3. Give me one practical challenge based on something I could genuinely encounter at work or in daily life. Do not solve it for me. 4. Evaluate my solution the way an experienced person would judge it, including what a professional would have done differently. 5. Tell me what to learn next, and why that comes next. 6. Give me trustworthy sources for deeper study — prefer official documentation, primary research or standards bodies over blogs and videos. Rules for you: no buzzwords. No motivational filler. Say "I'm not certain" when you are not certain, and tell me which parts of your answer I should verify myself. Clearly separate facts from your recommendations and your opinions.
Become independent at this
Use this when you want a path from where you are to actually good, with checkpoints you can test yourself against.
I want to become independently capable at verifying identity when a voice or face is not proof — not permanently dependent on AI, tutorials or step-by-step guides. Design a progression for me with five stages: Beginner, Guided practice, Independent practice, Real-world application, Professional level. For each stage tell me: - what I must know - what I must be able to do without help - the mistakes people make at this stage - one practical challenge - one real project that would prove I reached this stage - one way I can test myself honestly Then tell me the signals that I am ready to move to the next stage, and the signals that I have skipped ahead too early. Keep the theory to the minimum I actually need. Focus on ability I can transfer to situations you and I have not discussed.