A cool track can be heard anywhere: in a shopping mall, in a cafe, and even from the window of a nearby car, standing in traffic. In order not to miss an unfamiliar song you like, it's enough to turn on the recognition app. The name of the composition and the name of the performer in them are given out by artificial intelligence in seconds. However, there is a thorough preparation behind such a rapid result: in order to quickly learn a melody, the program first needs to memorize it. To do this, neural networks are introduced to a huge library of tracks, and then algorithms convert the sound into a spectrogram and decompose it into time, frequency and intensity.
A spectrogram is a graph. Time is located horizontally along the axis, the frequency of sound is located vertically, and its intensity at a fixed moment is expressed in color. A low signal is represented by a red stripe from below, and a high signal is represented from above. The result is a picture consisting of colored horizontal stripes. Analysis of such schemes helps to recognize music. When working with spectrograms, the same neural network approaches are used as in image analysis.
Let's say a person hears a song on the radio and wants to know the name and artist. The recognition program builds a spectrogram of the sounding passage and sends it to its library of tracks. Then he compares the "picture" of the desired melody with the spectrograms of other compositions and selects the most accurate match. At the same time, artificial intelligence recognizes the melody even through serious interference such as traffic noise or repairs in the neighboring apartment.
By the way, the neural network is able not only to identify the artist and the name of the track stuck in the head, but also to roughly determine its genre. To do this, artificial intelligence is taught to find patterns of different musical styles. Such specific characteristics are usually inaccessible to human vision and hearing. But thanks to machine learning, it becomes possible to calculate musical genres from spectrogram images.
It seems that finding the "right" track for the mood in billions of songs on your own is almost as unlikely as falling in love with each other at first sight. But thanks to recommendation algorithms, perfect matches are not so rare. First, artificial intelligence searches for people with similar tastes, and then statistical formulas are connected: the number of likes, dislikes, listenings and skips of a particular composition.
The recommendation of songs works according to a simple scheme: if Vasya liked track X, and then Petya also appreciated it, then when Vasya likes Y, it's worth recommending track Y to Petya. When the algorithm needs to find the next song, the ready formula is applied to a set of potential compositions. The most suitable one floats up.
"Cold" content, not seen in the playlists of the mass listener, spreads more slowly. But thanks to neural networks, unknown performers and niche music still have a small chance to flash in the stream of recommendations. If we simplify all the technical nuances, then we can say that in such cases, artificial intelligence finds out how often a particular user listens to songs with similar spectrograms, and periodically invites him to get acquainted with new tracks.
Sometimes I look for inspiration in recommendations. I trust the choice of composition to the music service, listen to the melodies, find interesting sounds or texts. So you can really spontaneously fall in love with a track by an unknown artist. And a line accidentally heard can prompt me to create my own poems.
Neural networks also help to generate collections of music for fitness, walking or sleeping. Content editors select reference tracks for algorithms, and based on their spectrograms, artificial intelligence expands thematic recommendations.
Previously, only composers could create melodies. Now it is possible without the participation of musicians. In 2020, the Netherlands hosted the Eurovision Song Contest for neural networks for the first time — AI Song Contest. The winner was the Australian collaboration of artificial intelligence with koalas, kingfishers and Tasmanian devils. The song was dedicated to the wildfires raging on the continent. Animal sounds were recorded in short samples — fragments 1-2 seconds long. The algorithm combined them with the hits of all the previous winners of the real Eurovision Song Contest, after which it collected samples into its own melody.
This is not the only example of a successful creative union of programmers and neural networks. In 2019, at the closing of the Winter International Festival of Arts in Sochi, the state orchestra performed an 8‑minute piece. It was written by composer Kuzma Bodrov from individual fragments of melodies generated by neural networks. Today, music creation is the most promising area for the development of artificial intelligence.
Artificial intelligence can create music in three ways. The first is related to the construction of ready—made "bricks" of sound - samples. In this case, the algorithm simply arranges them in the right order along several audio tracks, and the electronic arranger brings the finished track together. The second way is to generate musical notes. It's like writing an instruction for a musician to play a finished piece according to it. And the third way is to record a "raw" audio signal. In this case, the neural network itself creates sound waves that are similar, for example, to Mozart or the Beatles.
By the way, neural networks can also write poems for songs. So far, such tracks sound rather strange, so songwriters should not worry about unemployment. In addition, the "computer mind" is devoid of feelings. He cannot delve into the emotional context and convey the experiences that forced the authors of the works to create.
Poetry and music are primarily about the soul, inner world, experiences, feelings and emotions of people. For example, the new track "Don't Burn Out" is my personal story, but it's also about everyone who goes to a dream and tries to figure themselves out. I don't think artificial intelligence will ever replace a living person in the music industry. But here you can get an interesting tandem "human — neural network". We already know dozens of examples when artificial intelligence helped composers create unique melodies. In fact, this is a new direction in the music world, which, I am sure, will have its own listener and audience in the future.
Artificial intelligence makes creativity accessible to everyone, and music helps it develop. To figure out how these two poles converge and affect each other, you can on "The lesson Numbers " from Yandex — "Digital art: music and IT". Together with the comic book characters, participants will learn how neural networks recognize and generate tracks and what technologies help in the work of music services familiar to us. During the lesson, students will try to guess the melody themselves from the spectrogram and assemble a playlist with recommendations.
