Reading time 13 minutes

Soundtracks have become such an interesting newspaper topic that even the composers are awarded an Oscar for their products. Of these, video game tunes get less attention, although they mainly do contribute to the game’s immersiveness².

Background

In the 1950s, programmer Jeff Hill created digital music recorded on the CSIRAC computer³. Released in 1972, Pong was the first arcade game with sound t a commercial success. In 1979, the first-ever digital audio disc was presented. This phenomenon was noted “chiptune” since the soundtrack was created by an audio chip, usually inside a game console. The first such devices had the very limited sound processors. It was most commonly made with 8-bit music. This 8-bit processor created the melody in real time. In the early 1980s, the IBM personal computer appeared, and a couple of years later, sound cards and synthesizer modules came out. Each module has its own function: some generate sound, others act as a sequencer, others are various sound effects that played and created music in several channels.

In the 1950s, programmer Jeff Hill created digital music recorded on the CSIRAC computer⁴. Released in 1972, Pong was the first arcade game with sound turned into a commercial success. In 1979, the first-ever digital audio disc was presented. This phenomenon was noted as “chiptune”, since the soundtrack was created via an audio chip, usually inside a game console. The first such devices had very limited sound processors. It was most commonly made with 8-bit⁵ music. This 8-bit processor created the melody in real time. In the early 1980s, the IBM personal computer appeared, and a couple of years later, sound cards⁶ and synthesizer modules[/A synthesizer module is an electronic instrument divided into separate components (modules, blocks)] came out. Each module has its own function: some generate sound, others act as a sequencer⁷, others are various sound effects that play and create music on several channels.

In 1999, when the first music cartridge⁸ for the Game Boy was released, an updated genre of 8-bit music appeared – post-chiptune. It sounded less monotonous and simple, but technically differed little from the first experiments with digital music. Today, 8-bit is a separate genre that exists at the center of the subculture. Thanks to some bands, such as Crystal Castles, this music got a second chance in the 2000s. The creators of such tracks use special programs – trackers. They imitate the sound of chips from old set-top boxes. And 8-bit styled music can be made using modern synthesizers and sequencers.

 

 

The monotonous sound did not clash with the simple pixelated graphics of the first Dendy and SEGA games. The monotonous melody was quickly remembered and made the game recognizable. For example, the theme from the Guinness World Records best-selling game, Super Mario, became a symbol of the first console games⁹ of the 1980s. The same thing happened with the Tetris melody, which evokes nostalgia for those born in the 1990s. It is noteworthy that the Russian folk song “Peddlers” was used as the basis for the soundtrack for Tetris, but it became known abroad only in the context of the game as “music from Tetris”.

When discs appeared, the CD partially displaced audio chips, as it allowed tracks to be recorded and transmitted in advance. Sound designers had control over the process, as they could evaluate the finished compositions before they were combined with the game. The track was played directly from the disc, so the limited amount of memory on the set-top box itself is not a problem. Initially, the creators could use well-known songs. By the time, the stricter the licensing became, the stronger the power of composers in the game industry got.

Details

During game development[¹⁰, it was difficult to turn music into an accompaniment to the player during a chase, gunfight, or secret mission. The sound design also includes background sounds that accompany the action, the audio interface (e.g. in the main menu or when in danger), and character voices.

Music has a particularly important role in evoking emotion for game players. Hence, it must be more adaptable than the soundtracks. The theme of the game should not only set the mood, it must match exactly with the pace of the gameplay¹¹. You can go through the same episode at different speeds, so writing a finished album from finished tracks will not work. And then the musical units accompanied by games are smaller, which means they better adapt to each player.

Most games are unlikely to have time for slow and harmonious scenes and soundtrack transitions. The transitions from one rhythm to another must be done in a couple of seconds, depending on the user. For balanced transitions between episodes of the game, there is a horizontal and vertical convergence. The first occurs quite abruptly, for example, changing the research theme of the battle. The second involves modifying one main theme by speeding it up or slowing it down, increasing or decreasing the volume, and adding instruments. This is more often used not for abrupt movement between locations or tasks, but, for example, as a hint on the way. So, as you approach the nightclub in the game Ratchet & Clank: Rift Apart, the sound of dance beats will increase, new parties are added.

Since the game requires dynamic adaptation, the amount of material that the composer needs to write also increases. If this is about 50-70 minutes in the film industry, that is, half of the total running time¹², then the duration of the passage of the game varies more, and the amount of sound accompaniment changes along with it. Hence, for The Last of Us II, composer Gustavo Santaolalla wrote about three hours of tracks.

Process

Music for a film or game is more frequently perceived as an accompaniment or an additional design element. Consequently, it seems paradoxical that work on soundtracks begins even before filming, at the stage of the script, or after the creation of a storyboard by game designers.

Once the composers have become familiar with the concept and the intended atmosphere of the game, they choose the direction in which they will develop the sound design. If this is vertical mixing, then you need to set triggers by two properties: at what parameter does the music change and how it changes. It can be affected by location, weather, time of day, level of fatigue or health of the character. Individual musical phrases in vertical mixing, on the other hand, are highly simplified, lack structure, and are sometimes too chaotic. However, this mixing variant, which is most similar to the ambient music genre, is well suited for travel, occasionally interrupted by short fights or tasks. With horizontal mixing, the composer and editors already work with whole compositions, which are divided into several scenes and alternate with each other. As a rule, their triggers are associated with an emotion or a central event in a game sequence. This is a less harmonious way of mixing, since the transitions between musical phrases can be sharp, and the melodies themselves are too long.

Sometimes the game’s sound design also uses stingers – stand-alone compositions that can appear in the middle of vertical mixing. They emphasize important player actions and evoke emotions quickly. Often, they come with replicas of the heroes with commentary on what’s going on.

Nowadays, it is not necessary to specify the time of the next knock of the wheel or the bell of the blade, or the place where the character will hear whispers or birdsong. To do this, there is procedural generation – an artificial intelligence system developed to generate a unique soundtrack for each game from a database of samples¹³ created by the composers.

Sounds are selected depending on the scene, location and pace of the game, as well as the state of the character. For example, Rise of the Tomb Raider used Intelligent Music Systems’ dynamic percussion¹⁴ system for sound design. And in the game No Man’s Sky, for which the band 65 Days of Static wrote a track for 50 minutes, this composition is used as a dataset for artificial intelligence, which selects pieces from there to create a unique track.

Forms

In addition to background accompaniment, music can become part of the gameplay. For example, in Legend of Zelda: The Ocarina of Time, the character can play the instrument to advance to the next level. Buttons with notes on the screen have become a direct element of the game mechanics. In Genshin Impact, which became popular last year, players also have the opportunity to play the Lyre of the Winds by pressing the note symbols. Here, a musical instrument is not needed to complete missions, but serves as an addition to the main gameplay. Sometimes you can not play the music with the character’s hands, but just listen to it. For example, in the arcade¹⁵ game Gris, where the heroine begins to sing at the player’s click, accompanied by an ambient melody.