Voice Cloning

Voice cloning is the use of artificial intelligence to build a synthetic model of a person's voice and generate new speech in that voice from a limited audio sample. The technique underpins commercial text-to-speech, film dubbing and accessibility software, and it has become one of the most consequential tools in fraud: the United States Federal Bureau of Investigation recorded US$893 million in AI-related scam losses in 2025, with voice cloning named among the main drivers.^[1][2]

Background

Neural speech synthesis emerged with the deep learning wave of the mid-2010s. WaveNet, presented by DeepMind researchers in 2016, showed that neural networks could generate raw audio waveforms far more naturally than earlier concatenative systems.^[3] A second wave began in 2023, when zero-shot models such as Microsoft's VALL-E demonstrated that a voice could be approximated from as little as three seconds of reference audio.^[4]

Commercialisation followed quickly. ElevenLabs, founded in 2022, raised US$500 million in February 2026 at an US$11 billion valuation, more than tripling its valuation in a year, and now markets enterprise voice agents alongside its speech models.^[5]

How It Works

A modern voice cloning system extracts a speaker representation — an embedding of timbre, pitch and speaking style — from reference recordings, then conditions a neural text-to-speech model on that representation. Two families of methods dominate: zero-shot cloning, which needs only a short sample and no training, and fine-tuned cloning, which trains on minutes to hours of audio for higher fidelity.

Because zero-shot cloning needs so little input, public speeches, podcasts, voicemails and social media clips all supply enough material for convincing forgeries. Streaming systems have also cut latency to fractions of a second, enabling cloned voices in live conversation.

Applications

Legitimate uses span dubbing and localisation for film and television, audiobook and podcast production, game character voices, and accessibility, including voice banking that preserves a person's voice for use after speech loss. Voice agents built on the technology now answer customer service calls in several industries. The growth of cloning has also raised questions about consent and compensation for professional voice actors, whose recorded work can be used as training data.

Fraud and Misuse

The FBI's Internet Crime Complaint Center logged 22,364 AI-related complaints in 2025, with reported losses reaching US$893 million; impersonation of family members in distress, fake executive instructions to employees and bypassing of voice-based verification were among the recurring patterns.^[2] Security researchers advise verification through a separate channel before acting on any voice-only request for money or access.

Regulation and Countermeasures

There is no single global regime for voice cloning. The European Union's AI Act requires disclosure when audio has been artificially generated, with transparency duties for deepfakes applying since 2 August 2026.^[6] Industry responses include consent policies for voice libraries, provenance metadata in generated audio, and watermarking research. Detection of cloned audio remains difficult, which has pushed institutions to rely less on voice alone for authentication.

>See Also

• Deepfake
• AI Watermarking
• ElevenLabs
• AI in Malaysian Banking
• MCMC Malaysia