Blog homepage

The Making of SoundSeed Grain

New Releases / Sound Design / Wwise Tips & Tools

We have wanted to build a general purpose granular synthesizer for Wwise for a very long time  due to the wide range of sounds it can produce, and the many ways it can be used in games.
I remember having made a granular synthesis-like patch in Native Instruments’ Reaktor for myself, several years ago. I used it a little, and then found that it could not do exactly what I wanted, so I added new features. And then added even more features. And again. And it ended up like this...
Picture1
A horrible wall of buttons (not Reaktor - Reaktor is cool, my design wasn’t)
Because granular synthesis can produce a wide variety of sounds, I think that it is very easy to design a granular synthesizer organically, so that each feature is piled up on top of the others in what ends up being a big giant mess.

Focus versus Flexibility

It is commonly accepted that an audio plug-in is usually more successful when it is efficient at doing one specific thing very well. For example, Crankcase Rev uses granular synthesis, along with very sophisticated signal processing in order to successfully emulate the sound of combustion engines. Their UX and choice of user controls is thought out to make these kinds of sounds easy to create. Obviously, we wouldn’t want to make a plug-in that specializes in combustion engines, as Rev already does it very well.
While it infringes the rule stated above about being focussed, we wanted to create a plug-in that would cover most applications that require granular synthesis, without necessarily being specialized in any of them. Hopefully, a critical mass of users will be able to tweak it so that it works for their applications. So, we started with this simple question: What do people really expect from granular synthesis for their game audio?
Picture2
Mock up of the simplest granular synth
Picture3
A later mock-up.

User Research

While attempting to design a synthesizer that can do everything without being specialized in anything, it became obvious that we needed to get our community of Wwise users involved.
We asked the question: “What would you use a (traditional) granular synthesizer for (in your work)?” Here’s a sample of answers we received:
  • “car engines, textured ambiences, rolling sounds, rain, fire”
  • “explosions, destruction, oneshot, crumbling sounds”
  • “for evolving ambience bed, for sci-fi or ‘semi-natural’"
  • "weird, unnatural sound"
  • “wavetable synth caps for making music...”
  • “implementing run-time / interactive ‘effects’ with lots of control. For example: ‘make the monster hit sound more grainy / scarce as the monster scales are ripped off its skin.’"
  • “generating some cool granular sounds that I'd want to bounce back into a DAW and edit”
Many said that they “did not know yet”, and one person even said that they couldn’t find any use for it!
Certainly, the use cases would span a wide range. So, reaching out to the community did not stop at gathering use cases.
  • We gathered written conversations we already had with designers about granular synthesis before even starting the design.
  • We carried out interviews in-house.
  • We built a quick prototype and brought it with us to GDC, where we hoped to find people interested in trying it.
  • We made a second prototype that we shipped to users and carried out follow-up interviews where they would share their thoughts and ideas with us.
  • We shipped an alpha to get more feedback.
It was a long process of learning, testing, and validating, and we gained valuable knowledge at every step of this process. Questions users responded to, and topics of conversations during interviews we had with them were centred around:
  • Use cases;
  • Workflow and pain points;
  • Specific questions about how features were presented;
  • Missing features.
Conversations would often go way beyond that!
Picture4
The first functional prototype (GDC)
Picture5
The prototype sent out to users.
While I could go into detail about a gazillion things that came out of these interviews, I’ll just mention one example to give you an idea about what this process is like. (Just in case you’d like to be involved next time!)

Example: Grain Window

One user’s ambition was to design a car engine.They explained to us how they expected to see a “window”, on top of the waveform, from which grains would be randomly picked up. Also, these grains should correspond to explosions in the original file. And the window itself should move with an RTPC that is tied to some game-driven RPM.
Picture6
Expressing allowed grain start positions with a window and markers.
A special feature “grain window” for selecting grain start positions would be appropriate for this kind of application but not for others such as time-scale effects and wavetable synths. The same behavior could be equivalently set up by using random modulators on the Position cursor, but it would be difficult to visualize how modulation would translate into such a window and impossible to have grains start at explosions.
Picture7
Alpha
So, while we decided not to add a special “Grain Window” feature, which would be optimal for this car engine use case, this concept and the discussion around it ended up inspiring the features Snap to Markers and automatic transient detection, and Grain Visualization (overlaid on the waveform view). We think these features would cater to a larger variety of use cases. You can see how these features, along with modulation of Position can be used to emulate a “grain window” here.
This is instead how you would set up the synthesizer to emulate a “grain window”:
  1. Assign the Position property to an RTPC, which you will bind to an appropriate game parameter, say, RPM. This RTPC defines the left boundary of the macro window.
  2. Assign the Position property to a Random+ modulator with some Amount. The chosen Amount defines the width of the macro window, in percentage of the file's duration.
  3. If the width of the window should change with the RPM, bind it to RPM using an RTPC.
You may force grains to always start at precise positions in the original sound, for example at each explosion, by adding markers and using the "Snap to markers" option. Automatic transient detection may save you time.
Picture8
2018.1.4, showing in action envelope visualizer, grain window, and live monitoring under sliders.
To conclude, SoundSeed Grain was shaped by continuous valuable user feedback throughout development. Other concrete examples of features we were able to tweak as a result include:
  • The Envelope Visualizer (center of the window), without which users would struggle to understand the interactions between grain duration, emission, and envelope shape.
  • The slider feedback (live monitoring under each slider), necessary to work with modulation.
Picture9
The Envelope Visualizer
 
Picture10
Slider with live feedback
We will certainly pursue this user-oriented approach for other plug-ins, so if YOU would like to be part of the discussion about the next one, by all means, register here!

Acknowledgment

Audiokinetic would like to thank everyone who participated in the design process, our wonderful community who gave us feedback, and, in particular, Tom Bible, Nicholas Bonardi, Mike Caviezel, and Andrew Quinn for their contributions and immense help towards figuring out where we needed to go! 

Before you go, here's a quick video on SoundSeed Grain!  

 

  Subscribe

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni leads research at Audiokinetic and has been focused on spatial audio, synthesis and audio coding. He holds a Master's degree in Electrical Engineering, and lectures on and writes about various audio topics such as interactive music, dynamic mixing with HDR, and more recently, object-based panning and ambisonics. Louis-Xavier has presented talks at AES for Games, and MIGS, amongst other industry events.

 @xbuffoni

Comments

Leave a Reply

Your email address will not be published.

More articles

Wwise CPU Optimizations : General Guidelines

Empowering audio creators also means transferring some game resource responsibilities into their...

23.5.2017 - By Adrien Lavoie

Earthworms Trigger Audio Using Wwise : 'We - The Common Body'

Inspired by space, human's relationship with nature, and nature's relationship with technology, the...

10.10.2017 - By INFER Project

Tuning your Game: Music Theory for Sound Design

During MIGS17, Vincent Gagnon, Audio Director at Ubisoft, gave a presentation on Music Theory for...

12.6.2018 - By Audiokinetic

Wwise 2018.1 is Live! Colours and Beyond…

Wwise 2018.1 is out and available for you to download from the Wwise Launcher. The new colourful UI...

6.8.2018 - By Audiokinetic

Ring of Elysium – The Sounds by Watson Wu

Watson Wu is an award winning American sound designer, field recordist and composer. Well known in...

16.4.2019 - By Watson Wu

Botanical Dimensions

In October of last year, my team and I helped put together an amazing experience for Intel in...

14.5.2019 - By Braeger Moore

More articles

Wwise CPU Optimizations : General Guidelines

Empowering audio creators also means transferring some game resource responsibilities into their...

Earthworms Trigger Audio Using Wwise : 'We - The Common Body'

Inspired by space, human's relationship with nature, and nature's relationship with technology, the...

Tuning your Game: Music Theory for Sound Design

During MIGS17, Vincent Gagnon, Audio Director at Ubisoft, gave a presentation on Music Theory for...