For most of photographic history, creating a convincing fake image or recording required substantial skill.
That assumption has collapsed.
Modern generative AI can create or manipulate:
- photographs;
- voices;
- video;
- music;
- faces;
- documents;
- entire scenes.
A person can appear to say words they never spoke.
A familiar voice can seemingly call a family member.
A photograph can depict an event that never occurred.
A real video can be altered so subtly that the manipulation is difficult to notice.
This technology is commonly discussed under the word deepfake, but deepfakes are only one part of a larger phenomenon:
synthetic media.
Synthetic media can be entertaining, artistic, educational and useful.
It can also be fraudulent, abusive and politically destabilizing.
The central problem is therefore not simply:
How do we stop fake content?
It is:
How do people establish trust when authentic and synthetic media can both look convincing?
The Short Answer
A deepfake is generally an AI-generated or manipulated image, audio recording or video that convincingly depicts a person, object, place or event in a way that falsely appears authentic.
Synthetic media is broader. It includes AI-generated content that may not be deceptive at all.
Examples include:
- a fictional AI-generated character;
- a translated voice track;
- a movie effect;
- a virtual spokesperson;
- an AI-generated training simulation.
The problem arises when synthetic content is presented in a context where people reasonably believe it documents reality.
The European Union's current AI Act guidance defines deepfakes around AI-generated or manipulated image, audio or video resembling real people, objects, places, entities or events and falsely appearing authentic or truthful. Its Article 50 transparency obligations began applying on August 2, 2026.
Deepfakes Are Not New—But Their Economics Are
Photographic manipulation predates computers.
Audio has long been edited.
Film effects have placed people in situations that never occurred.
What generative AI changes is the combination of:
cost + quality + speed + accessibility.
A capability that once required:
- specialized software;
- trained artists;
- expensive hardware;
- significant time;
can increasingly be performed through a consumer interface.
NIST's current deepfake research describes the creation of highly realistic manipulated media as increasingly low-cost and accessible.
The threat is not that deception was invented.
The threat is that high-quality fabrication is becoming cheap enough to scale.
Voice Cloning Changes Impersonation
Audio deserves particular attention.
Humans are accustomed to treating a familiar voice as a strong identity signal.
If your daughter calls, you recognize her.
If your boss speaks, you recognize him.
Voice cloning weakens that assumption.
Generative systems can reproduce characteristics of speech and produce new sentences in a similar-sounding voice.
The FTC has repeatedly warned that voice cloning can strengthen impersonation scams and has treated technical countermeasures as only one part of the solution.
The safest response is behavioral:
A familiar voice is no longer sufficient authentication for an unusual request involving money, credentials or secrecy.
Impersonation Fraud Is Already Enormous
The broader impersonation problem is economically significant even without attributing every case to AI.
The FTC reported that consumers said they lost $3.5 billion to imposter scams in 2025, with nearly one in three fraud reports involving impersonation. The FTC does not claim that all—or even most—of these losses involved deepfakes or AI, so the figure should not be presented as an AI-fraud total.
AI matters because it gives impersonators additional tools:
- realistic voices;
- convincing profile photographs;
- personalized messages;
- synthetic video;
- scalable translation;
- faster social engineering.
The scam itself may be old.
The impersonation layer becomes more convincing.
Deepfakes Can Be Harmful Without Involving Money
Financial fraud receives attention because losses are measurable.
But synthetic media creates other forms of harm.
A person can be falsely depicted:
- making a statement;
- committing an act;
- appearing in intimate content;
- endorsing a product;
- attending an event;
- supporting a political position.
The harm may be reputational, emotional, professional or political rather than financial.
Once content spreads, proving that something is false may not fully repair the damage.
Politics Creates Two Problems, Not One
The obvious political danger is fabricated evidence.
A fake recording appears to show a candidate saying something outrageous.
But there is another danger.
Once deepfakes become widely known, real evidence can be dismissed as fake.
This produces a credibility problem:
“That recording isn't real. It's AI.”
Society therefore faces both:
false media accepted as real
and:
real media rejected as false.
The second problem is sometimes overlooked.
A world full of synthetic media can weaken trust even when no particular deepfake succeeds.
Detection Is an Arms Race
One proposed solution is AI that detects AI.
That is useful.
It is not sufficient.
Detection systems search for statistical or forensic signs of manipulation.
But generation systems improve.
Content can be recompressed.
Images can be cropped.
Video can be re-recorded.
Adversaries can deliberately attack known detection techniques.
NIST's 2026 deepfake-evaluation work reports major performance degradation when some detection systems move from academic-style evaluation to operational conditions and is developing more adversarially realistic benchmarks for this reason.
This means a detector score should not be treated as unquestionable proof.
The Alternative: Prove Where Media Came From
Another strategy begins with a different question.
Instead of asking:
Can we detect that this image is fake?
ask:
Can we establish how this image was created and changed?
This is the idea behind content provenance.
The Coalition for Content Provenance and Authenticity—C2PA—develops technical standards for attaching cryptographically protected information about the source and history of digital media. Its current 2.4 specification was released in April 2026.
That information can record things such as:
- how an asset originated;
- which tools modified it;
- which transformations occurred;
- whether credentials remain valid.
This is often surfaced to users as Content Credentials.
Provenance Does Not Mean “True”
This distinction is critical.
A provenance record can help establish:
where something came from
and:
what happened to the file.
It does not automatically prove:
the claim depicted in the file is true.
An authentic photograph can be accompanied by a false caption.
A genuine video can be taken out of context.
A real quotation can be selectively edited.
Provenance is therefore an important trust signal, not a universal truth machine.
Missing Credentials Do Not Mean Fake
The opposite mistake is equally important.
A photograph without Content Credentials is not necessarily synthetic.
Billions of existing cameras, images and workflows do not yet create or preserve provenance information.
Metadata can also be stripped by platforms or transformations.
The correct interpretation is therefore:
credentials present and valid → useful evidence about provenance
not:
credentials absent → fake.
Regulation Is Moving Toward Disclosure
The European Union has made transparency a legal part of its AI framework.
Starting August 2, 2026, Article 50 obligations cover machine-readable marking of certain AI-generated or manipulated outputs and disclosure obligations for deepfakes and some AI-generated public-interest content.
The EU also finalized a voluntary Code of Practice designed to help providers and deployers demonstrate compliance with those mandatory transparency requirements.
This reflects an important shift.
The objective is not to ban synthetic media generally.
It is to make relevant artificial generation or manipulation visible enough for people and systems to interpret appropriately.
Synthetic Media Has Legitimate Uses
The term deepfake can make every synthetic image sound malicious.
That would be a mistake.
Generative media can support:
- film production;
- localization;
- accessibility;
- education;
- games;
- historical visualization;
- privacy-preserving avatars;
- virtual production;
- advertising;
- satire;
- artistic experimentation.
A film actor could speak naturally in another language.
A teacher could create a historical simulation.
A game could generate individualized characters.
A performer could license a digital representation.
The technology itself is not equivalent to deception.
Intent, consent and context matter.
Consent May Become a Central Digital Right
When AI can reproduce a person's appearance and voice, identity becomes a reusable digital asset.
That raises questions society did not previously need to answer at this scale.
Who can create a synthetic version of you?
Can your employer?
Can an advertiser?
Can a fan?
Can your family after your death?
Can you license your voice for one project without authorizing every future use?
Synthetic media therefore connects technical AI questions with:
- privacy;
- publicity rights;
- copyright;
- contract law;
- employment;
- consumer protection.
The Verification Habit Has to Change
For decades, “seeing is believing” was a useful shortcut.
It is becoming less reliable.
That does not mean people should believe nothing.
Universal skepticism is not a solution.
Instead, high-stakes media requires better verification.
Ask:
Where did this originate?
Find the earliest credible source.
Is the account authentic?
A real-looking profile is not enough.
Is there corroboration?
Important events usually produce multiple independent records.
Does the file have provenance information?
Content Credentials can provide useful evidence where available.
Is someone creating urgency?
Fraud often relies on panic.
Can the claim be verified through another channel?
Call the person or organization using a number you already know.
Family Verification Codes Are Becoming Practical
A simple technique can defeat sophisticated impersonation.
Families can agree on a private verification phrase.
If someone calls claiming to be a family member and urgently needs money, ask for the phrase.
A cloned voice cannot automatically know information never made public.
Businesses can apply the same principle using existing authorization procedures.
AI makes process more important than intuition.
Journalism Faces a New Burden
News organizations increasingly need to verify not only what a source says but whether digital evidence is authentic.
That can involve:
- source verification;
- metadata;
- geolocation;
- forensic analysis;
- provenance;
- cross-source comparison.
The result may be a paradox.
AI makes false media cheaper to create.
That makes trustworthy verification more expensive.
Five Misconceptions About Deepfakes
“Deepfake means any AI-generated image.”
Synthetic media is broader. A fictional generated image does not necessarily impersonate reality.
“Humans can always spot a fake if they look closely.”
Visual quality continues improving, and many manipulations are designed specifically to avoid obvious artifacts.
“AI detectors can solve the problem.”
Detection remains useful but has important operational limitations.
“Content Credentials prove that something is true.”
They provide evidence about provenance, not the truth of every claim represented.
“If media has no Content Credentials, it is fake.”
Absence of provenance information is not proof of fabrication.
What Technology Platforms Can Do
Platforms can combine several defenses:
- provenance support;
- machine-readable labels;
- deepfake detection;
- impersonation reporting;
- account verification;
- rapid removal for certain harmful material;
- preservation of original metadata.
No single layer is sufficient.
The strongest systems will use several.
What Individuals Can Do
Slow down
Urgency helps scammers.
Verify through another channel
Do not trust the communication itself to prove who sent it.
Use private authentication information
Especially for family and financial requests.
Check provenance when available
But interpret it correctly.
Avoid spreading sensational material immediately
A few minutes of verification can prevent large amplification.
What to Watch Next
Real-time deepfakes
Live synthetic audio and video will make “but we were on a video call” a weaker form of identity verification.
Provenance adoption
C2PA succeeds only if cameras, creative tools, platforms and users preserve credentials through the content chain.
Legal identity rights
Rules around voice, face and digital likeness will continue developing.
Detection benchmarks
Operational testing will matter more than laboratory accuracy.
Agent-powered scams
AI agents could combine research, impersonation and automated communication into more scalable fraud workflows.
The Bottom Line
Deepfakes create a problem larger than fake videos.
They challenge one of the basic shortcuts people use to understand the world:
I saw it, therefore it happened.
The answer cannot be permanent distrust.
It also cannot depend solely on detecting every fake after it has been created.
The emerging trust system will likely combine:
detection
provenance
disclosure
authentication
law
human verification habits.
The future of trustworthy media may depend less on asking:
Does this look real?
and more on asking:
Where did it come from, what happened to it, and what independent evidence confirms it?
Questions People Ask
What is a deepfake?
A deepfake is AI-generated or manipulated audio, image or video that convincingly depicts something in a way that may falsely appear authentic.
Are deepfakes illegal?
The answer depends on the jurisdiction, context and use. Existing fraud, impersonation, privacy, harassment, election, intellectual-property and other laws may apply even where there is no single general “deepfake law.”
Can deepfakes be detected?
Sometimes, but detection is an ongoing technical challenge and performance can decline under real-world or adversarial conditions.
What are Content Credentials?
They are a way of presenting provenance information based on standards developed by C2PA, allowing digital media to carry cryptographically protected information about its origin and modification history.
Does the EU require deepfake disclosure?
Certain AI Act transparency requirements applying from August 2, 2026 include obligations relating to deepfake disclosure and machine-readable marking of specified AI-generated or manipulated content.