Please be advised that the Audiokinetic offices will close at 3:00 p.m. EST on December 23rd, 2019 for the holiday period, and will reopen on January 3rd, 2020 at 9:00 a.m. EST. Learn more.
Blog homepage

Doin' it Beatnik-style: MIDI + samples in Wwise

Interactive Music

The first time I looked at the MIDI music system in Wwise, I was struck by how functionally similar it was to the now-obsolete Beatnik engine, a software synthesizer I used many years ago to produce game audio and interactive music. This is a prime example of the adage: "ya live long enough, ya become cool again". 

beatnikBeatnik engine

I had just started to work with the Wwise MIDI system to reduce the size of a mobile game music soundtrack, when the gig disappeared in a sudden flash of dragonfire! Undaunted, I still wanted to try it out, because the MIDI+samples method of producing interactive music provides tools for creative invention and adaptive implementation unavailable to streaming media systems. 

So I wrote "metal(notHeavy)" using the same techniques I would have had it been a Beatnik track. In fact, I wanted to demonstrate a variety of techniques available to composers when controlling sample playback using MIDI data. As the project evolved, a creative theme emerged as well, specifically "all sounds produced by metal": glockenspiel, agogo bell, steel strings (piano, bass), vibes, brass, et al. 

Dowload the "metal(notHeavy)" Wwise Project

Recording Samples 

I started with the piano loop. This is a live recording of my 1926 Steinway baby grand using a PianoMic system by Earthworks. The physics behind this remarkable mic setup seems like voodoo magic to me, but to my ears the results are astonishing. I played to click, and cut the loop in ProTools. 

Then I recorded the glockenspeil (an Orff instrument recently retrieved from an attic in Massachusetts). It only ever had C diatonic keys, plus a couple of F#'s and Bb's; I recorded each metal bar, and would fill in the gaps later to create a chromatic instrument. 

glockenspielCarl Orff (Carmina Burana) was not only a composer of apocalyptic full-orchestra-and-choir pieces, he also developed a pedagogy for teaching music to very young children (kindergarten through 4th grade). The Orff Schulwerk method relied on rhythmic clapping, pentatonic scales, and special mallet instruments, including various sizes of xylophones (wood keys), metallophones (aluminum, aka vibes), and this glockenspiel (steel).

I once took an "african bell" class at UC extension, and learned a variety of patterns: guanguanco, kidi, african 6, agogo, timbale. I recorded multiple takes of these rhythms using a rubber mallet and a Neumann KM184, then cut up four bar phrases in ProTools. Full disclosure, I did apply some quantization to these samples, sliding audio hits around by hand to line up exactly to tempo (100 bpm). 

The rest of the samples were gleaned from various sources over the course of the project: additional recordings, synthesizers, and sample libraries. They are all in blend containers in the "instruments" actor-mixer, formatted as 16/48 WAV (the full-rez 24/96 recordings of the bell patterns are also available to download, for anyone interested). 

Triggering Samples 

A Wwise MIDI music segment works just like a regular one, except you drop in .mid rather than .wav files. Each track is associated with a instrument blend container, which contains a keymap defining note ranges and roots for each sample (again, like Beatnik). 

agogobell cabasa sawbladeAgogo Bell (left) Cabasa (center) Sawblade (right)

There's a few ways the samples are triggered: 

  • The piano loop is triggered once every 20 bars. 
  • The bell patterns are shorter, and are triggered more often. Each MIDI note is assigned to a random container of loops, sorted by catgegory. The MIDI sequence defines which categories play when, but the bell pattern mix will vary on subsequent playbacks. 
  • The glockenspeil is laid out so each steel bar sample is associated with the corresponding MIDI note. Then the ranges of the C, D, and G notes are stretched to cover the C#'s, Eb's, and Ab's of the chromatic scale. 
  • The bass and vibes instruments are similarly laid out, except that I recorded the MIDI first, then looked at the range of notes played, and took samples at regular intervals within that range. This produces smooth tonal transitions between high and low notes. 
  • The cabasa, ride cymbal, and sawblade effect are all one-note hits, associated with containers of variations to prevent annoying sonic repetition. 
  • The brass is an assortment of phrases culled from an old school sample CD. They've been cut into short phrases, and are triggered in different combinations and timings to create melodic riffs. The technique is designed to produce increased sonic bang for your audio budget buck. 

Fun with MIDI 

Putting a song together in this manner results in a dramatic reduction of memory usage. In fact, an .mp3 rendition of the piece is double the size of the Wwise Vorbis bank (2.3Mb). The project demonstrates how small audio chunks can be played and manipulated via MIDI to create large amounts of variable musical content. This is an especially useful technique when producing soundtracks for resource constrained platforms, like mobile and wearable devices. 

metal(notHeavy)_wwiseProjclick on the screenshot to enlarge 

But simple and straightforward it ain't! I could have much more easily and quickly produced the same song (sans variable bell patterns) in ProTools by just laying down tracks and rendering a .wav file. And if you're under a tight deadline (because the programmer needs to integrate the music now now now), then yeah, no problem, here ya go! 

However, this project was produced soley for my own creative edification, and I must admit, I very much enjoyed writing without consideration to what the stakeholders might think, or whether the music fit the game theme. My point is, doing purely personal projects after your job goes up in smoke can be an effective way of channeling the inevitable layoff blues into something privately satisfying (that might even help you find work in the future). 

In any case, halfway through development, while playing with my audio toys and working out the technical implementation, I realized that what I was really doing was just having FUN ... the most important aspect of any creative project! 

MP3 rendition of the song metal(notHeavy)

- pdx

Peter "pdx" Drescher

Peter "pdx" Drescher

Peter "pdx" Drescher produces audio content, implementation, and code for games and mobile devices. He is a talented sound designer, accomplished musician/composer, published author, experienced programmer, and skilled audio engineer. His ProTools project studio "Twittering Machine" is located in the SF Bay area.

www.twittering.com/

Comments

Ken Felton

August 20, 2019 at 12:00 pm

Hey PDX, Nice overview. Thanks for sharing! Cheers- Ken

Garry Schafer

August 20, 2019 at 01:37 pm

Hey I thought I was the only guy on this planet that remembered and loved Thomas Dolby’s Beatnik — used it on numerous projects in Director and it was exactly like having a little Akai sampler with a General Midi synth; there was nothing else like it for Director, and, when I started getting into Unity, nothing like it for Unity - until now. Great article; thanks for the memories!

Leave a Reply

Your email address will not be published.

More articles

On Composing Interactive Music: let's time-travel to 1993...

In 1993, I was working on my master's degree at New York University's Interactive...

5.12.2017 - By John Szinger

MUR: Interactive Music for a Children's AR Book & Game on Tablet! 

Creating truly interactive music can be rather hard. One of the reasons is the nature of music....

6.2.2018 - By Jesper Ankarfeldt

Behind the Music of Halo Wars 2: An Interview with Composers Brian Lee White, Brian Trifon and Gordy Haab

One of the most buzzed about video games of 2017 has been Halo Wars 2 with The Guardian calling it...

20.2.2018 - By Amie Cranswick

Is Hybrid Interactive Music the Future? PART II - Technical Demonstrations

In part one of this blog, we discussed Hybrid Interactive Music and why it is so important that we...

4.4.2018 - By Olivier Derivière

Total War: WARHAMMER 2: How we construct and systematically use our music

For those of you who may not be familiar with the Total War series of video games, Total War is a...

14.8.2018 - By Creative Assembly

The Differences Between Working in Game Audio and Film: Part 1

Have you been working in the film industry and are wondering what it would be like to work in game...

18.6.2019 - By Jesper Ankarfeldt

More articles

On Composing Interactive Music: let's time-travel to 1993...

In 1993, I was working on my master's degree at New York University's Interactive...

MUR: Interactive Music for a Children's AR Book & Game on Tablet! 

Creating truly interactive music can be rather hard. One of the reasons is the nature of music....

Behind the Music of Halo Wars 2: An Interview with Composers Brian Lee White, Brian Trifon and Gordy Haab

One of the most buzzed about video games of 2017 has been Halo Wars 2 with The Guardian calling it...