AI Jukebox creates ‘deepfake’ songs, imitating dead pop stars

Artificial intelligence (AI) is used to create new “ deepfake ” pop songs that appear to be performed by dead musicians including Elvis Presley, Frank Sinatra, David Bowie and Michael Jackson.

Jukebox, created by the Californian company OpenAI, is a neural network that generates weird approximations of pop songs in the style of several artists.

The neural network generates music, including rudimentary chants with English lyrics and a variety of instruments like guitar and piano.

Experts believe the technology has the potential to create royalty-free knockoffs or even new tracks that could be released as singles under the name of a deceased artist.

Search Jukebox Below

OpenAI has created a vast library of new tracks, emulating a diverse selection of artists, including The Beatles, Nirvana, Katy Perry, Simon and Garfunkel, Stevie Wonder, Elton John and Ed Sheeran, as well as deceased heroes who almost seem to be brought in. back to life.

Most of the samples have a weird and distant quality to them, as if they were poorly produced demos from the 1950s that hadn’t seen the light of day until now.

Not all of them can reasonably be mistaken for the imitated artist – an entry listed as “rock in the style of the Beatles” sounds like a bad impersonation of Radiohead’s Thom Yorke.

The song begins with the lyrics “Without leaving my home I can know everything about Earth” and fades to an unsatisfying conclusion.

But the entry under “glam metal in the style of darkness” could be mistaken for a new single from the flamboyant British rockers.

OpenAI says there has been rapid progress in creating controversial deepfake videos and generating text and images using AI technology, but artificial music is advancing as well.

“We are presenting Jukebox, a model that generates music with vocals in the raw audio realm,” said the OpenAI research group in their article titled Jukebox: A Generative Model for Music.

“The large-scale combined model can generate high-fidelity and diverse songs with consistency of up to several minutes.

“We can condition on artist and genre to direct musical and vocal style, and on unaligned lyrics to make singing more controllable.

“We publish thousands of unsolicited samples, along with weights and model code.”

OpenAI retrieved 1.2 million songs, of which 600,000 are sung in English, from the Internet and associated them with lyrics and metadata, which were fed into the AI ​​to generate approximations of the different artists.

The firm said its models could produce songs from various musical genres including rock, hip-hop and jazz while capturing “ melody, rhythm, and timbre. ”

“As an engineering, it’s really impressive,” Dr. Matthew Yee-King, electronic musician and computer expert at Goldsmiths, told The Guardian.

“ They break down an audio signal into a set of music lexemes – a dictionary if you prefer – at three different time levels, giving you a basic set of fragments that is enough to reconstruct the music that was introduced.

“The algorithm can then rearrange those fragments based on the stimulus you enter.

“So give him some Ella Fitzgerald for example, and he’ll find and put together the relevant pieces from the dictionary to create something in his musical space.”

Experts believe the technology could change the music industry by creating new successes, but it risks causing music copyright issues.

Deepfake Music blurs the line between using a copyrighted song and using a cheaper or non-copyrighted approximation.

“If someone hasn’t used the recording themselves, you won’t have any legal action against them in terms of copyright in the sound recording,” said Rupert Skellett, manager. the British record company Beggars Group.

Creating deepfake music isn’t new – in January, Microsoft announced a new collaboration with Icelandic singer Björk to create a series of musical compositions with a custom artificial intelligence tool.

The AI ​​created new variations of Björk’s original arrangements based on changes in weather conditions and the position of the sun.

Described as a “ generative soundscape, ” the composition combines sounds and patterns from Björk’s personal choir archives, which she has compiled over the past 17 years as a solo artist.

In 2017, scientists created a “ Bot Dylan ” computer capable of writing its own folk music.

The system uses artificial intelligence to compose new tracks after being trained using 23,000 pieces of Irish folk music.

Study author Dr Oded Ben-Tal, senior lecturer in music technology at Kingston University in London, said: ‘We weren’t expecting any of the machine-generated melodies is very good.

“But we, and several other musicians we’ve worked with, were really surprised at the quality of the music created by the system.

“People are reluctant to believe that machines can be creative – it’s seen as a very human trait.

In 2017, Spotify hired French AI expert and composer François Pachet from Sony, fueling speculation that the streaming giant is embarking on artificial music creation.