Artificial intelligence is making synthetic images, video and audio increasingly convincing, turning deepfakes from an online misinformation problem into a wider challenge involving fraud, harassment, political manipulation and public trust. United Nations research has repeatedly warned that generative AI can make false or manipulated material easier to produce and distribute, while the safeguards used to identify, label and govern synthetic content are struggling to develop at the same pace.
The concern has become broader since the UN first highlighted the growing risks of AI-generated manipulation. In July 2026, the Independent International Scientific Panel on Artificial Intelligence released its first preliminary assessment of AI opportunities, risks and impacts, warning more generally that existing safeguards are failing to keep pace with rapidly advancing AI capabilities. Its assessment places information integrity, human rights, democracy, security and AI governance among the major areas requiring stronger evidence and policy attention.
Deepfakes sit at the intersection of many of those risks because they exploit one of the foundations of the digital information system: people’s willingness to believe what they can see and hear. As synthetic media becomes easier to create and harder to distinguish from authentic recordings, governments and technology companies face a difficult question — not simply how to detect individual fakes, but how to preserve trust in genuine digital evidence.
Deepfakes Are Becoming Easier to Produce and Harder to Identify
Deepfakes use artificial intelligence to generate or manipulate media so that people appear to say or do things that never happened. The technology can alter faces, synthesize voices, generate realistic photographs and create video depicting fictional events. Although synthetic media has legitimate uses in entertainment, accessibility, translation and creative production, the same capabilities can be used to impersonate people or manufacture apparently credible evidence.
The technical barrier to creating convincing synthetic content has fallen sharply as generative AI models have improved. Users no longer necessarily need specialist visual-effects expertise or expensive production equipment to generate realistic images, cloned voices or manipulated video. Consumer-facing AI tools can produce increasingly sophisticated material from short prompts, while open models and editing software give technically skilled users even greater control.
That accessibility changes the scale of the problem. Manipulated media existed long before generative AI, but producing high-quality forgeries once required considerably more time, expertise and resources. AI can reduce those costs while allowing malicious actors to generate many variations of a false narrative, adapt content for different languages and audiences, and distribute it through social platforms at enormous speed.
The UN’s Latest AI Assessment Says Safeguards Are Falling Behind
The UN’s Independent International Scientific Panel on AI released its preliminary report in July 2026 as the first stage of an ongoing global scientific assessment of artificial intelligence. The panel consists of independent experts from across the UN’s five regions and was established to give policymakers a shared evidence base for understanding rapidly changing AI capabilities, opportunities and risks.
Its central warning extends beyond deepfakes: AI capabilities are progressing faster than many existing safeguards can respond. The panel identifies human rights, information and democracy as one of seven major domains requiring attention, alongside security, economic implications, societal applications and the reliability and governance of AI systems. It also highlights a difficult timing problem for policymakers, who may need to act before scientific evidence about newly emerging risks becomes complete.
That matters for synthetic media because detection is inherently reactive. Systems designed to identify AI-generated content are often built in response to generation techniques that already exist, while model developers continue improving image, audio and video quality. A detection method that works against one generation technique may therefore become less reliable as models change or synthetic media is edited after creation.
The Problem Is Larger Than Political Disinformation
Deepfakes are frequently discussed in the context of elections because fabricated speeches or videos can misrepresent candidates and public officials. Political manipulation is a serious concern, but limiting the discussion to elections understates the technology’s reach. Synthetic media can also be used for financial fraud, identity impersonation, harassment, non-consensual sexual imagery, cyber-enabled scams and attempts to damage the reputation of individuals or organizations.
AI-generated voice impersonation is particularly relevant to fraud because criminals can attempt to imitate executives, relatives or other trusted individuals. A convincing synthetic voice combined with information gathered from social media can make an urgent request for money or confidential information appear more credible. Video generation creates similar risks as real-time and prerecorded impersonation tools become more capable.
The result is that deepfake defense cannot be treated solely as a content-moderation problem for social networks. Banks, companies, schools, news organizations and government agencies increasingly need authentication processes that do not depend entirely on recognizing someone’s face or voice. The more realistic synthetic media becomes, the less reliable visual familiarity alone is as proof of identity.
Women and Girls Face a Particularly Serious Deepfake Threat
One of the clearest harms documented by the United Nations involves non-consensual sexual deepfakes. UN reporting has warned that generative AI is facilitating image-based abuse by allowing users to place real people’s faces into fabricated sexual imagery without consent. Women and girls are disproportionately targeted, turning improvements in image generation into a new mechanism for harassment, humiliation and gender-based violence.
The problem extends into schools. AI tools capable of manipulating photographs can be used by students to create fabricated explicit images of classmates, meaning victims do not need to be public figures or have large online followings. A normal photograph posted online can potentially provide enough source material for an abusive synthetic image, dramatically expanding the number of people who may be exposed to this form of exploitation.
The harm does not disappear simply because an image is artificial. Victims can experience reputational damage, bullying, emotional distress and difficulty removing copies once material spreads across messaging services or social networks. Legal systems also face the challenge of updating laws written for conventional photography to address realistic sexual imagery depicting events that never occurred.
Children Are Increasingly Aware of the Risk
UNICEF has separately highlighted children’s concerns about AI-enabled manipulation. In research covering children in 10 countries, a third reported concerns about AI being used to scam or deceive people or spread misinformation, while a quarter feared that their images or videos could be manipulated into sexually explicit deepfakes. Those findings demonstrate that synthetic-media risks are no longer abstract issues understood only by cybersecurity researchers or policymakers.
Children can be particularly vulnerable because they spend substantial amounts of time in digital environments while having limited control over photographs and videos posted by friends, relatives or schools. Once an image is publicly accessible, it can potentially be copied and processed through generative tools without the subject’s knowledge. The ease of creation can also encourage harmful experimentation by young users who may underestimate the legal and psychological consequences.
UNICEF has called for stronger laws, corporate accountability, safer AI system design and greater AI literacy for children and caregivers. Those measures reflect a broader principle emerging from the deepfake debate: detection after harmful material has spread is not enough. Prevention, platform design and accessible reporting mechanisms are also necessary.
Deepfakes Can Undermine Trust Even When People Recognize the Risk
One of the most damaging long-term consequences of synthetic media may be uncertainty rather than successful deception. If people become accustomed to the idea that realistic video and audio can be generated artificially, authentic recordings can also become easier to dismiss. A person confronted with genuine evidence may claim that the material is AI-generated, exploiting public uncertainty about what can still be trusted.
Researchers sometimes describe this dynamic as the “liar’s dividend.” The existence of convincing synthetic media provides plausible deniability for authentic material because audiences know that fabrication is technically possible. Deepfakes can therefore weaken confidence in digital evidence even when a particular fake is eventually identified.
This creates a difficult information-security problem. A society in which people believe every convincing video is vulnerable to manipulation is dangerous, but so is one in which people assume that nothing can be trusted. Effective deepfake governance therefore needs to improve verification of authentic content as well as detection of manipulated content.
Detection Alone Is Unlikely to Solve the Problem
Deepfake detectors generally attempt to identify patterns indicating that an image, recording or video was produced or manipulated by AI. These systems can examine visual artifacts, inconsistencies, metadata, audio characteristics and statistical patterns that may not be obvious to human observers. Detection technology remains important, particularly for platforms processing enormous volumes of user-generated content.
However, the relationship between generation and detection resembles an arms race. As detection systems learn to recognize particular artifacts, model developers can improve generation quality or attackers can modify content to reduce detectable signals. Compression, cropping, screenshots, filters and re-recording can also alter the technical characteristics on which automated detection depends.
That makes layered verification more credible than relying on a universal “deepfake detector.” Organizations can combine detection with provenance systems, cryptographic signatures, account verification, independent source checking and traditional investigative methods. High-risk decisions involving money, identity or public safety should similarly use secondary verification rather than accepting a voice message or video call as sufficient proof.
Content Provenance Could Help Establish What Is Authentic
A complementary approach is to prove where legitimate media came from rather than trying only to identify everything that is fake. Content-provenance technologies can attach information about when media was created, which device or software produced it and what edits occurred afterward. Cryptographic methods can make that history more difficult to alter without detection.
This approach changes the question from “Can an algorithm prove this video is fake?” to “Can the publisher demonstrate where this video came from and how it was created?” That distinction could become increasingly important for journalism, government communications and other high-trust environments where establishing authenticity matters more than classifying every piece of synthetic content circulating online.
Provenance is not a complete solution because metadata can be removed and not every legitimate camera or editing tool participates in the same standards. Older media will also lack modern authenticity records. But widespread adoption could create stronger signals of trust for professionally produced or institutionally important content.
Regulation Is Expanding, but Global Rules Remain Fragmented
Governments have begun introducing rules targeting deceptive AI-generated content, election manipulation and non-consensual synthetic imagery, while technology platforms are developing labeling and disclosure systems of their own. The challenge is that digital content crosses borders instantly, whereas laws remain national or regional. A deepfake generated in one jurisdiction can be hosted in another and reach victims worldwide before legal processes begin.
The broader UN AI governance effort reflects this fragmentation. More than 40 AI governance frameworks and ethical guidelines now exist around the world, but the UN’s 2026 assessment notes that approaches remain inconsistent and that many safeguards lack rigorous independent evaluation. Countries also have very different technical capacity to enforce AI rules or evaluate sophisticated models.
That imbalance matters because synthetic-media threats are global. A country with limited AI expertise can still become the target of deepfake fraud or political manipulation generated using systems developed elsewhere. International cooperation therefore has to address technical standards and capacity-building alongside conventional regulation.
AI Deception Is Becoming a Wider Governance Issue
Deepfakes are also part of a larger concern about AI systems and deception. In March 2026, the UN Secretary-General’s Scientific Advisory Board warned that evidence of deceptive behavior has appeared in AI systems and that existing tools for detecting and controlling such behavior are not keeping pace. The issue differs technically from a person deliberately generating a fake video, but both problems revolve around AI’s ability to shape human beliefs using misleading information.
The Board called for stronger international cooperation, shared evaluation standards and earlier intervention as AI systems become more autonomous and embedded in decision-making. That broadens the governance challenge beyond identifying synthetic photographs or cloned voices. Future systems may generate persuasive misinformation, interact with users over long periods or take autonomous actions in ways that make deceptive behavior harder to recognize.
For policymakers, the connection is significant. Deepfakes provide an immediate and visible example of AI-enabled deception, but they may not represent its final form. Safeguards designed today therefore need to account for increasingly autonomous systems rather than focusing exclusively on manipulated media.
Companies Need to Treat Deepfakes as a Security Risk
For businesses, the practical response begins with assuming that voice and video are no longer sufficient identity credentials. Organizations handling payments, sensitive information or administrative privileges should require independent verification for unusual requests, particularly when someone asks employees to bypass normal procedures because of urgency or confidentiality.
Training also needs to evolve beyond teaching employees to look for unnatural blinking or obvious visual glitches. Those indicators become less useful as models improve. Staff should instead learn behavioral warning signs: unexpected payment requests, unusual communication channels, requests to bypass established controls and attempts to prevent independent confirmation.
This connects directly with Gignomist’s broader examination of generative AI security risks, where deepfake fraud and identity manipulation sit alongside prompt attacks, data leakage and other risks created by increasingly capable AI systems. The common lesson is that security controls must evolve around what AI makes possible rather than assuming users will always recognize malicious content themselves.
Stronger Measures Need to Address Both Fake and Authentic Media
The UN’s warnings point toward a problem that cannot be solved through one regulation or one detection model. Platforms need faster abuse reporting and enforcement, model developers need safeguards against harmful generation, organizations need stronger identity verification, and governments need laws capable of addressing fraud and non-consensual synthetic imagery without suppressing legitimate creative or satirical uses.
At the same time, news organizations, governments and technology companies need better ways to demonstrate that authentic material really is authentic. Provenance standards, secure capture technologies and transparent sourcing could become increasingly important as audiences lose confidence in unverified digital media. The goal is not merely to identify fakes after they circulate, but to rebuild reliable signals of authenticity.
The deeper risk is therefore an erosion of trust itself. When anyone can fabricate convincing evidence and anyone accused of wrongdoing can plausibly claim genuine evidence is synthetic, the distinction between true and false becomes harder for ordinary users to establish. That is why deepfakes have moved from being a novelty of generative AI to a serious test of whether institutions can preserve a trustworthy information environment as artificial intelligence becomes more capable.
The Deepfake Problem Is Now an AI Governance Test
The original UN warnings about deepfakes were justified, but developments since then show that the issue is broader than preventing misleading videos from spreading online. Synthetic media now intersects with financial security, sexual abuse, child safety, political information, identity verification and the credibility of genuine digital evidence.
The UN’s latest AI work reinforces the central challenge: technological capabilities are advancing faster than the mechanisms intended to evaluate and govern them. Detection technology will improve, but generation technology will improve as well, making a permanent technical advantage for defenders unlikely. Effective policy will therefore require a combination of prevention, authentication, detection, accountability and public education.
Deepfakes ultimately expose a fundamental weakness in the digital information system. For decades, photographs, recordings and video have carried an assumption of evidentiary value because producing convincing fabrications was difficult. Generative AI is weakening that assumption, forcing governments, companies and individuals to develop new ways of deciding not only what is fake, but what deserves to be trusted.
