0
Views
Collect a playlist, find a track stuck in your head, write a play: what artificial intelligence can do with music
Miscellaneous / / March 30, 2022
Recognize compositions
A cool track can be heard anywhere: in a shopping center, in a cafe, and even from the window of a nearby car, while standing in a traffic jam. In order not to miss the unfamiliar song you like, it is enough to turn on the recognition application. The name of the composition and the name of the artist in them are given out by artificial intelligence in seconds. True, behind such a rapid result there is a thorough preparation: in order to quickly learn the melody, the program first needs to remember it. To do this, neural networks are introduced to a huge library of tracks, and then algorithms convert the sound into a spectrogram and decompose it into time, frequency, and intensity.
Anatoly Starostin
Head of Technology Development Service at Yandex Media Services.
A spectrogram is a graph. Time is located along the horizontal axis, the frequency of sound is located along the vertical axis, and its intensity at a fixed moment is expressed in color. A low signal is represented by a red bar at the bottom, and a high signal at the top. The result is a picture consisting of colored horizontal stripes. The analysis of such circuits helps to recognize music. When working with spectrograms, the same neural network approaches are used as in image analysis.
Suppose a person hears a song on the radio and wants to know the name and artist. The recognition program builds a spectrogram of the sounding passage and sends it to its library of tracks. Then it compares the "picture" of the desired melody with the spectrograms of other compositions and selects the most accurate match. At the same time, artificial intelligence recognizes the melody even through serious interference, such as road noise or repairs in a neighboring apartment.
By the way, the neural network is able not only to identify the artist and the name of the track stuck in the head, but also roughly determine its genre. To do this, artificial intelligence is taught to find patterns in different musical styles. Such specific characteristics are usually inaccessible to human sight and hearing. But thanks to machine learning, it becomes possible to calculate musical genres from spectrogram images.
Recommend Songs
It seems that finding the "same" track to suit your mood in billions of songs on your own is almost as unlikely as falling in love at first sight. But thanks to recommendation algorithms, perfect matches don't happen all that often. First, artificial intelligence looks for people with similar tastes, and then statistical formulas are connected: the number of likes, dislikes, plays and skips of a particular composition.
Anatoly Starostin
Song recommendation works according to a simple scheme: if Vasya liked track X, and then Petya also rated it, then when Vasya likes Y, Petya should also recommend track Y. When the algorithm needs to find the next song, the formula is applied to a set of potential songs. The most suitable floats to the top.
"Cold" content, not seen in the playlists of the mass listener, spreads more slowly. But thanks to neural networks, unknown artists and niche music still have a small chance to flicker in the stream of recommendations. If we simplify all the technical nuances, then we can say that in such cases, artificial intelligence finds out how often a specific user listens to songs with similar spectrograms, and periodically invites him to get acquainted with new ones tracks.
Mary Gu
Singer.
Sometimes I look for inspiration in recommendations. I entrust the choice of the composition to the music service, listen to the melodies, find interesting sounds or texts. So you can really spontaneously fall in love with a track by an unknown artist. And another line I accidentally heard can prompt me to create my own poems.
Neural networks also help generate music selections for fitness, walking or sleeping. Content editors select reference tracks for algorithms, and based on their spectrograms, artificial intelligence expands thematic recommendations.
generate music
Previously, only composers could create melodies. Now it is possible without the participation of musicians. In 2020, the Netherlands hosted the first Eurovision Song Contest for neural networks - the AI Song Contest. The Australian won collaboration artificial intelligence with koalas, kingfishers and tasmanian devils. The song was dedicated to the forest fires raging on the continent. Animal sounds were recorded in short samples - fragments of 1-2 seconds long. The algorithm combined them with the hits of all the previous winners of the real Eurovision, after which they assembled the samples into their own melody.
This is not the only example of a successful creative union of programmers and neural networks. In 2019, at the closing of the Winter International Arts Festival in Sochi, the State Orchestra performed an 8-minute piece. It was written by the composer Kuzma Bodrov from separate fragments of melodies generated by neural networks. Today, the creation of music is the most promising area for the development of artificial intelligence.
Anatoly Starostin
Artificial intelligence can create music in three ways. The first is connected with the construction of ready-made "bricks" of sound - samples. In this case, the algorithm simply arranges them in the right order over several audio tracks, and the electronic arranger mixes the finished track. The second way is to generate music notation. It's like writing instructions for the musician to play the finished work on it. And the third way is to record the "raw" audio signal. In this case, the neural network itself creates sound waves that are similar, for example, to Mozart or the Beatles.
By the way, neural networks can also write poetry for songs. So far, such tracks sound rather strange, so songwriters should not worry about unemployment. In addition, the "computer mind" is devoid of feelings. He cannot penetrate into the emotional context and convey the experiences that forced the authors of the works to create.
Mary Gu
Poetry and music are primarily about the soul, inner world, experiences, feelings and emotions of people. For example, the new track “Don't Burn Out” is my personal story, but it is also about everyone who goes after a dream and tries to understand themselves. I do not think that artificial intelligence will ever replace a living person in the music industry. But here you can get an interesting tandem "human - neural network". We already know dozens of examples when artificial intelligence helped composers create unique melodies. In fact, this is a new direction in the music world, which, I am sure, will have its own listener and audience in the future.
Artificial intelligence makes creativity accessible to everyone, and music helps it develop. To understand how these two poles converge and influence each other, you can "Lesson Numbers” from Yandex - “Digital Art: Music and IT”. Together with the heroes of comics, the participants will learn how neural networks recognize and generate tracks and what technologies help in the work of music services we know. At the lesson, students will try to guess the melody by the spectrogram themselves and compile a playlist with recommendations.
I want to "Lesson Numbers"
Cover: Willyam Bradberry / Shutterstock / Eric Isselee / Shutterstock / Ljupco Smokovski / Shutterstock / FOTOSPLASH / Shutterstock / Olga Selepina / Lifehacker
Subscribe
YOU MIGHT ALSO LIKE
