(Bloomberg) -- ElevenLabs, a startup that uses artificial intelligence to clone people’s voices, is working with an AI detection service to combat the rising threat of deepfakes during a crucial election year.
The startup has partnered with Reality Defender, a US-based company that offers deepfake detection capabilities to governments, officials and enterprises. The partnership is part of ElevenLabs’ broader efforts to boost safety on its platform, and follows concerns from researchers earlier this year that its technology was misused to create deepfake audio of US President Joe Biden.
The tie-up gives Reality Defender access to ElevenLabs’ data and models, allowing it to better detect AI-generated audio. In return, ElevenLabs will be able to use Reality Defender’s tools to boost its own safeguards against misuse. A spokesperson for ElevenLabs said there was no financial component to the deal.
The partnership highlights the growing risk of video, audio and image-based deepfakes. Creators can use generative AI tools to portray individual victims in false scenarios, such as deepfake porn. There are also concerns about the use of deepfakes for political manipulation during a historic year for elections globally.
In January, voice-fraud detection company Pindrop Security Inc. concluded that doctored audio of Biden that circulated to thousands of voters in New Hampshire had been created using ElevenLabs’ technology. The deepfake’s creator was suspended from the startup’s service, Bloomberg News reported.
“Deepfakes in 2024 have proven to be the newest iteration of document forgeries and compromising material used to target public figures and political leaders,” Clement Briens, a senior threat intelligence analyst at cybersecurity company Recorded Future, told Bloomberg News. “Heads of state, elected officials and journalists figure amongst consistent targets for deepfakes globally, and in a majority of 2024 elections worldwide.”
ElevenLabs said it had released additional safety features, such as automatically blocking voices of political figures, personal voice verification and a tool that detects whether a voice sample is AI-generated. The company is committed to boosting its own safety standards and making AI content detection easier, and is “focusing on this intensely during this vital election year,” said head of AI safety Aleksandra Pedraszewska in an interview.
Broader worries about safeguards have coincided with a steady uptick in venture capital funding to startups that offer powerful tools for creating synthetic media.
ElevenLabs, founded in 2022 by former Google engineer Piotr Dabkowski and Palantir alumnus Mati Staniszewski, has raised more than $100 million and was valued at $1.1 billion in January. Its voice cloning tech is targeted at users who, for example, may want to replicate their own voice in a different language. While it prohibits cloning a person’s voice without their permission, experts remain concerned by potential misuse.
Reality Defender was founded in 2021 and is a graduate of the Y Combinator startup program. The company says it has worked with banks, media companies, nonprofits and government officials.
--With assistance from Mark Bergen.
(Updates with further context from the second paragraph)
©2024 Bloomberg L.P.
