Future Timeline

Generating realistic audio requires modeling information represented at different scales. For example, just as music builds complex musical phrases from individual notes, speech combines temporally local structures, such as phonemes or syllables, into words and sentences. Creating well-structured and coherent audio sequences at all these scales is a challenge that has been addressed by coupling audio with transcriptions that can guide the generative process, be it text transcripts for speech synthesis or MIDI representations for piano. However, this approach breaks when trying to model untranscribed aspects of audio, such as speaker characteristics necessary to help people with speech impairments recover their voice, or stylistic components of a piano performance.

In “AudioLM: a Language Modeling Approach to Audio Generation”, we propose a new framework for audio generation that learns to generate realistic speech and piano music by listening to audio only. Audio generated by AudioLM demonstrates long-term consistency (e.g., syntax in speech, melody in music) and high fidelity, outperforming previous systems and pushing the frontiers of audio generation with applications in speech synthesis or computer-assisted music. Following our AI Principles, we've also developed a model to identify synthetic audio generated by AudioLM.

New AI image-generation systems make headlines every day but that revolution started many years ago. Now one of the most established services for AI face generation has expanded its offering to include full body images. An early use of computer-made faces was for news stories, video games, and documentaries when a person was needed to convey an idea or represent an unknown individual for which no photo was available. Keeping a stock library of faces isn’t too difficult for an agency but standing poses are harder since the type of clothing affects the possible uses of the images. In the past, one or more models would need to be hired for these types of shots.

The music industry’s lobbying arm claims that services using machine learning to alter tracks are infringing on artists’ rights.

As first reported by TorrentFreak, the Recording Industry Association of America listed AI-powered music websites that make remixes, improve homemade tracks, or strip songs of vocals or instrumentals harm artists, in a response to a request from the Office of the US Trade Representative.

Artists working within all kinds of media have raised concerns in recent years—and increasingly, with the rising popularity of text-to-image generators like DALL-E—about whether AI-generated art infringes on individuals’ copyright. Most AI content generators depend on datasets that are filled with original artworks, texts, or audio, and use those original works without the owners’ permission.