Blog homepage

Is Hybrid Interactive Music the Future? PART II - Technical Demonstrations

Game Audio / Interactive Music

In part one of this blog, we discussed Hybrid Interactive Music and why it is so important that we continue thinking about and developing a much wider range of use for music in games.  We also discussed some of the creative thinking behind the music for Get Even. Now, in part two of this blog, I will provide you with Get Even Hybrid Interactive Music technical demonstrations.

While I created various systems for the music of Get Even,  I will explore just one of them below, as it is already quite elaborate! 

SYSTEM #1 - Real-Time Generated Music

In the very first level of Get Even, you find yourself in an abandoned building. Your goal is to find a kidnapped girl. SPOILER AHEAD - Your main character and the other players do not know that they are going to almost die in the process. When the players get to the girl there is a bomb strapped to her and they can’t avoid the explosion, whatever they do. As makers of the game, we also know that the main character is reliving this traumatic experience, and we thought that their subconscious holding this memory, should make this experience almost unbearable.  

6-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

In Get Even, you have to save the girl.

As I mentioned in part one of this blog, I needed a diegetic starting point for the music. I thought about the various room tones as an idea. The player goes through a succession of rooms and corridors, so this made sense. So, we created all the room tones in C (in 3D, each room had at least 4 sound emitters to create room tones). We also added some buzzing light sounds, spread all over the level (each with its own sound emitter), and here too we managed to tune them to the key of C. These 2 elements were the pillars of the textural music. I created a drone in C (in 2D) with different layers that would blend with the room tones in the beginning, then get more distorted with the buzzing light sounds, as the players get closer to the girl. It was very effective, but we wanted to go even deeper and make it even more meaningful. 

The ticking time bomb is a huge part of this game. I thought a sound of a ticking clock could very much be the signature of the game, so to add another layer of meaning and tension it by making the ticking accelerate the closer you get to the girl. This is when I saw the light! What if the ticking clock would alter the perception of the main character, aka the player? What if this virtual reality, based on an actual memory (the character's memory), would tell you more than what it seemed to convey, since your character subconsciously knows this sound leads them to their own death? 

So, the sound of the clock should alter your surroundings. How do we do that?

For those of you familiar with dance music, I bet you understand the principle of side-chaining. Wikipedia's explanation: 'A compressor with a side-chain input controls gain from main input to output based on the level of the signal at the side-chain input. The side-chain input is used by DJs  lowering the music volume automatically when speaking.' 

So, I used this technique, but in my own way. Rather than using side-chaining for the music, I used it on the ambient sounds of the game. So, if it were raining, or if it was windy, or for the room tones, or any sound emitter in the game, I could / or could not make them go through a side-chain. 

The magic behind side-chaining in Wwise

First you want to use the effect Meter. Here is a great tutorial by Audiokinetic, explaining how to use it.

So here is my “Pumping_off_beat” RTPC:

 

7-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

RTPC Pumping_off_beat

I then applied a BUS. Here, a generic BUS (pumps) some curves for volume and a low-pass-filter. This makes the BUS output softer when the RTPC  gets any signal.

  

8-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

 

I wanted to have full control on what would create the side-chain, so I used Wwise Synth One and generated a very basic sine sound with control over its ADSR.

 

9-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

 A simple sine with RTPC on its ADSR

 
 
10-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

RTPC controlling the sine ADSR (N.B the scale does not show proper values)

I also wanted to have control over the intensity of the pump separately.  So, I created an RTPC that would allow me to make the pumping very subtle or very over-the-top, and I called it Pump_ON_OFF_Beat_Intensity.

 

11-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

RTPC that controls the intensity of the pumping created with side-chain

I was then able to trigger the sine in MIDI to side-chain it on any BUS. You can see below that the sine would be triggered every 2 bars at 120 BPM = 1 bar at 60 BPM.

 

12-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

The trick now is to make the sine sound disappear from the audio, as you don’t want to hear a beep every time you side-chain. So, I added a Wwise gain on the Pump bus and made it go to -96 dB. You can also see in the screenshot below that I created different types of side-chains. There is one for 'regular' use, which I show, but I also created an inversion of that. When the sine plays, any BUS that is side-chained will play and stop when the sine stops. It was very interesting because I could make any looping sounds such as buzzing lights, drones, water leaks and more, play a rhythmic pattern. I also created another side-chain, but for this blog, let’s just stick to the 'basic one' . The wonderful thing was that I could switch from one type of side-chain to another on the fly, using one RTPC (Switch pump).

 

13-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

3 different types of side-chaining and the gain control over the sine volume

 

14-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

RTPC that allows me to switch side-chain type

The 1 Bar that made my day

Now that you understand the principle of my use of side-chaining, let’s get to an actual example.

The pattern in the screenshot below is looped. It is only 1 bar at 60 BPM (2 bars at 120 BPM) = clock ticking. I added an RTPC to control the tempo called Tempo_prologue. The closer you get to the girl, the higher the value becomes. This RTPC acts on several objects:

  • The speed of the music
  • The drone intensity
  • The clock sound and verb

While playing you have room tones (in C), light buzzing (in C),a drone that builds-up (in C), some steam pipes that get all pumped in sync with a sound of a clock, and the closer you get to the girl, the faster, the bigger the intensity, and the more realistic the sound of the ticking clock becomes. Here is how I did it:

 

15-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png
  • In blue, the music sequence of 1 bar at 60 BPM (2 bars at 120 BPM)
  • In green, you can see the sine that is triggered in MIDI (real-time generated)
  • In pink, you can see all the elements that are under side-chaining. The BUS for ambient sounds, the sound FX (that loop), the drone.
  • In yellow, it’s about the ticking clock in MIDI (Sampler)

Below, you can see how the drone intensifies in sync with the tempo

16-Interactive Music Wwise - Olivier Deriviere - Get Even.png.png

Drone sound that becomes more intense with Tempo_Prologue RTPC getting faster

For the ticking clock it was a little tricky. I wanted the sound to start very pitched down with a big reverb, and the faster the tempo (the closer the player gets to the girl), the more realistic (higher pitched) and closer the sound becomes. So I created a MIDI instrument using Sampler. I could trigger the clock with MIDI but I had to create different starting points as, depending on the speed of the sound, the downbeat would be respected. I used a Switch Container and applied the Tempo_Prologue RTPC on it, and in the music editor I added switch tracks that would play depending on the tempo. It is amazing!!!

 

17-Interactive Music Wwise - Olivier Deriviere - Get Even.png

 

18-Interactive Music Wwise - Olivier Deriviere - Get Even.png

Switch container controlled by Tempo_Prologue RTPC that swaps switch tracks and makes the clock in-sync with the downbeat.

 

19-Interactive Music Wwise - Olivier Deriviere - Get Even.png

Blend Container of the clock sound, to have a better definition of the low pitch sound controller by Tempo_Prologue RTPC

 

Finally I added a reverb to give the clock, when in low tempo, an ominous feel. Once again, depending on the Tempo_Prologue, the verb will progressively disappear.

 

20-Interactive Music Wwise - Olivier Deriviere - Get Even.png

Reverb applied on the clock sound controlled by Tempo_Prologue RTPC

In the end, with just one RTPC (Tempo_Prologue) and 1 bar at 60 bpm, I was able to make the musical experience completely organic for the whole level. This is also great because if the developer were to change the scale of the level, the music would continue to follow the progression as the RTPC is dependent on the distance of the player to the girl. 

This is one of the many systems I created for Get Even, and I think the result is quite convincing when you play the game. This was also great for production. As the music is based on game parameters and generated in real-time, I didn’t have to worry about any changes that could happen during the production of the game.

 

21-Interactive Music Wwise - Olivier Deriviere - Get Even.png

 24 strings from The Brussels Philharmonic complemented the Real-Time Generated Music

Conclusion

In part one of this blog, I discussed how I believe interactive music can be more than what we perceive it is. The second part of the blog may leave you wondering about the frontiers between sound design and music, and this is a legitimate question. I happen to think that as soon as you deal with harmony and tempo, you enter the world of music. Going with real-time generated music opens a brand new world when it comes to producing music for games and how the player can experience it. I wrote one hour of live music with some very emotional cues performed by the Brussels Philharmonic, and I had a lot of fun blending them with all the systems that I created. I also composed a lot with the MIDI tool using Sampler in Wwise. I went as far as I could, and I would encourage anyone curious, to play the game and discover each of the systems. Here is a video of some of them. 

I would like to give credits to sound designers Filip Hajzer, Chris Grant and David Guinot, who spent nights with me making it possible! You rock guys!

I very much want to thank The Farm 51 for their amazing open mind, Bandai Namco for their strong support, and of course Audiokinetic for the help they provided me with and the incredible tool they created!

To all the pioneers out there: THANK YOU! 

Get Even has been nominated by the IFMCA (International Film Music Critics Association) for BEST ORIGINAL SCORE FOR A VIDEO GAME OR INTERACTIVE MEDIA, was a part of the CLASSIC FM TOP 5 SCORES of 2017, and was also nominated at Develop Awards 2017 for BEST MUSIC DESIGN.

Olivier Derivière

Composer

Olivier Derivière

Composer

For more than a decade, Hailed by MTV as “Game music’s eclectic daredevil”, IFMCA award-winning composer Olivier Derivière (Remember Me) has been scoring music for animation, films and video games and is widely acclaimed for both his composition and innovative approaches to connecting music with the game experience, including the groundbreaking titles, Assassin’s Creed IV Freedom Cry and Remember Me. He is recently done with Vampyr the next game by Dontnod Entertainement (Life is Strange).

www.olivierderiviere.com

 @oderiviere

Comments

Leave a Reply

Your email address will not be published.

Also from this series

Is Hybrid Interactive Music the Future? PART I - How I used Get Even as an R&D platform for Interactive Music

When I write music for video games, I always wonder about how I can make it more meaningful for the...

27.3.2018 - By Olivier Derivière

Also from this series

Is Hybrid Interactive Music the Future? PART I - How I used Get Even as an R&D platform for Interactive Music

When I write music for video games, I always wonder about how I can make it more meaningful for the...