Halo, and the birth of cinematic adaptive music

The release of Halo: Combat Evolved (2001) was a landmark moment in gaming; pushing boundaries both technically and creatively, and casting the mold for first person shooters for decades to come. It set new standards in word-building, cinematic storytelling and immersive gameplay, and the iconic soundtrack was a fundemental part of that. It is a score that mirrors the scale and emotional impact of film music, using powerful themes and motifs while embracing and innovating adaptive techniques.

When Halo first came out, I was immediately impressed by how successfully it created the sense of a personal film score, underscoring your actions and reflecting changes to your situation. It showed that videogame soundtracks could be grand, emotional and full of character, music with cinematic ambition that could also be interactive and adaptive without compromising on scale or impact.

Martin O’Donnell was the Audio Lead and composer (with additional music by Michael Salvatori), and was kind enough to answer some questions about his approach to adaptive music.

Before you started working on your first game projects, how aware were you of adaptive systems/techniques in videogame music? 

I’ve been playing computer/video games since the 70’s but was never interested in writing music or doing audio for them until I played Myst in 1993. That was the first game, for me, that seemed to approach interactive entertainment with a film aesthetic. At that point most of the music in games was not really adaptive and once I started working on my first two projects, Riven and Myth, I saw the opportunity for creating music and audio that could enhance and support stories and gameplay in real-time.

What led to you adopting one program/system over another?  What were the limitations you came across that led you to develop a custom system for Halo 1 onwards?

I don’t remember that there were any programs or systems in 1996 that would allow me to do the kind of adaptive audio work that I thought would be useful. The tools that Bungie made for audio were already moving in the direction that I believed could make effective use of randomization, layering, and interacting with the players. Developing a custom system for the first Halo was our only real option.

Were there any games back then that you took as inspiration for their adaptive techniques?  Are there any now that you find impressive or inspiring?  Are there any aspects or tropes of adaptive music that you find over or underused?  

The late 90’s and early 2000’s was an amazing time for all of the composers in the industry because there really wasn’t much precedence for us to build upon. I traded some ideas and techniques with the folks at LucasArts but pretty much kept my head down and figured out my own way of solving the issues. Today, I think that most game developers should let composers decide how to implement music rather than instruct the composers how to write music for their programmers. I feel like I can always tell when programmers or level designers have the final say on music in the game. The choices they make are just too obvious; “Walking around” music, or “battle” music. Music that increases in tension the same way at the same events – every time. It doesn’t reflect the aesthetic that we can learn from the great film scores. Also, music can be used as a crutch for games that are generally weak – which leads to an overabundance of music. Use too much music and it will lose its impact and the ability to affect the player’s emotions.

I’ve heard you speak about the importance of creating music that supports and enhances gameplay without drawing attention to itself.  But I know that many composers struggle with the problem of feeding the player with vital information while retaining that subtlety.  For example, a change in state from relaxed/exploration to danger/battle can in some cases benefit from being abrupt and obvious.  Do you ever struggle with striking that balance?

Of course – all the time. I look at the score to Jaws by John Williams to remind me of how to play with those emotional state changes. He sets the audience up many times with anticipatory music but then at the moment the shark fully reveals itself – no music. In a game, being abrupt and obvious can help the player, but there should be times when you go against expectations and allow surprise or emotional juxtaposition. Once the score becomes completely predictable, in a movie or a game, it either disappears or becomes annoying.

As an audio director, can you talk about the relationship between the role of sound effects and music respectively when it comes to providing important information to the player?  i.e. does music always provide emotional information, with sfx used as a more immediate feedback tool?  Has that balance changed for you from project to project?

I have a catchphrase that I’ve used for many years “Sound makes it real, music makes you feel”. Of course, that’s just a general rule of thumb and there are many times and places where it doesn’t hold true. Especially with sound effects which can be great at evoking emotions. Different kinds of games will require a different balance. Strong story driven games will probably rely more on music to enhance the emotional journey of the player, but games that are more ethereal or surreal can use some amazing atmospheric sound design to create changing emotional moods. Puzzle, sports, and music games are in a different category of course.

You are often credited with bringing epic and cinematic music to video games in a way that changed the genre.  Were there times when you found the interactive requirements of a score inhibiting your cinematic ambitions?

What usually happens for me is that I compose music that works on its own. Music that tells its own story using traditional compositional and structural techniques. Then I dissemble it to see how it can be implemented adaptively. About the only kinds of pieces that don’t work well are the ones that steadily increase in intensity and volume. There are times that call for simpler, more ambient pieces that can be used as single colors or moods and allow for more abrupt responses to changes in gameplay by overlaying something completely different. I really try to approach the game as if it were a linear score and then break up those music changes into pieces that can adapt to the variations of gameplay.

Nintendo are regarded by many as pioneers and innovators when it comes to adaptive music and use of musical sound effects.  The abstract, surreal nature of those worlds seems to help- games set in more realistic or cinematic environments are more restrictive in what their audiences will accept from the music.  Do you find that to be the case?  

For me the difference in approach comes from trying to be “real” or trying to be “gamey”. There are definite times in any game experience, menus or collectables for example, when the sound effect needs to communicate directly and clearly. The player needs to know instantly something that advances the game itself – like picking up ammo or making a successful selection on a menu screen. For me, music that becomes “gamey” is annoying. “Victory” music or “Death/Failure” music quickly becomes something that I’d rather turn off because it takes too long or is too repetitive to communicate something I probably already know. A music game or a stylized animated game can use sounds and music that would seem out of place in a more cinematic, realistic game environment.

Film music has developed into an established language of tropes and techniques that audiences have come to expect, and which composers exploit or subvert to great effect.  Would you say that video game music has matured to that same extent?

Certainly, we have reached a point where aesthetically and technically video game music has matured. The tools and platforms we have now allow us to compose and implement just about anything we can imagine. Film music seems to get into a rut every few years and then someone comes along and breaks out. I think the same applies to game music. We are only limited by our imaginations.

I’ve noticed that VR games that provide high levels of musical interactivity and feedback (e.g. rhythm action games) seem to be hugely popular at the moment.  It’s as if the immersion of VR and the immersion of interactive/adaptive music are perfectly suited for one another.  Aside from the more technical aspects, have you found yourself approaching Golem’s sound and music in a different way than you would a non-VR game?

On the sound design side absolutely. Musically however, I’m still approaching it from the emotional journey side of things. Knowing where the player’s ears are and having the ability to immerse people in a fully 360-degree sound environment is amazing. If music is non-diegetic, not something that plays in the environment, then I believe it’s best to have the music essentially “inside” the players head, enhancing and informing the emotions. I’m looking forward to hearing how other composers and designers solve these issues. And I’d love to work on a music game sometime.