When it comes to the development of audio plug-ins, many sound designers think of it as a “black magic” that is exclusive to audio programmers because it usually requires coding skills as well as good knowledge in math, physics and digital signal processing, skills not absolutely essential for sound designers.
When sound designers hear of DSP effects, not available in Wwise, from someone else’s projects, they are probably able to understand how they work very quickly, and create technical prototypes with their commonly-used tools to reproduce the same features. However, due to either performance limitations or the difficulty for the output of such tools to be deployed as plug-ins in the authoring tool or engine, most ideas stop at the prototype stage and fail to benefit real games.
Luckily, with PureData and Heavy, sound designers now have the opportunity to independently build proper effects or source plug-ins that can be used in the authoring tool and run in real games.
What is PureData?
PureData (PD) is a visual programming language, developed by Miller Puckette in the 1990s, used to create interactive music and multimedia works. PD is an open-source project with a large developer base working on new extensions. And you can now even load VST plug-ins in it!
The file containing the PD code is called a Patch. Also, PD supports SubPatches. Users can create complex PD projects with modular units of code.
Although, this is not a PD starter tutorial. For those who want to know more, please go to https://puredata.info/.
As you can see from the above video, PD is used in much the same way as a modular synthesizer: building a signal system by connecting different objects and interacting in real-time with it by adjusting parameters.
PD can be considered a “blueprint environment” for sound designers in which to create technical prototypes with their commonly-used tools, then generate plug-ins with a proper compiler.
What is Heavy?
Heavy is a PD Patch compiler, developed by Enzien Audio, that can convert the PD Patches created by sound designers into C/C++ code. This allows sound designers to work in the PD environment to generate code that can run on the target platform without having to write them manually.
Heavy can be considered as an “automatic workshop” that can produce things with the “blueprint” created by sound designers.
So, what can we do with the product?
What can we do with the plug-ins created using PD + Heavy?
1. Perform real-time DSP operations based on the sound designers’ design in the PD Patch;
2. Provide support for the input and output of audio. If the source Patch contains an adc~ object, it will generate an effect plug-in. If not, it will generate a source plug-in (such as Wwise SoundSeed Wind, Wwise SynthOne).
3. Provide support for the input and output of control parameters. It can either be controlled by the Game Parameters from the Wwise Sound Engine or send data to them, to be used as a LFO, for example, among other things.
4. Provide support for the input and output of UI events (Unity only).
Now that we know what sound designers can do with these tools, let’s try to produce something with them.
Sample Divider effects are usually included in a distortion plug-in like Bitcrusher. I’d like to reproduce such features with PD + Heavy.
Why we would want to attemp this:
1. Wwise doesn’t have a Sample Divider plug-in;
2. Sample Divider is actually very useful, especially when Cyberpunk becomes popular again (let’s say you want to get the similar malfunction effects that are frequently occurred in Nier: Automata and Titan Fall 2, how can you do it without Sample Divider?).
Worry-free DSP Processing
Before creating a “blueprint” using the Patch file, we need to understand how Sample Divider works.
Sample Divider resamples the input signal with a lower frequency than its sample rate, then distorts the original signal with lower output quality.
The following image illustrates how the Sample and Hold (S/H) module works (retrieved from The Theory and Technique of Electronic Music, by Miller Puckette).
In the image shown above, the sine wave signal is sent to the IN point of the S/H module, while the sawtooth wave signal is sent to the OUT point to trigger the Sample & Hold operations. When the sawtooth wave values drops down to 0, it will trigger the S/H object to resample the input signal while keeping the sampled values unchanged for the remaining duration. In this way, the output signal is more similar to the input signal when the sawtooth wave frequency comes close to the sample rate of the input signal. Otherwise, there will be more distortion.
Here is a demo video that shows how the S/H module works in VCVRack (similar to a physical synthesizer).
Creating a Blueprint
Now that we know how the S/H module works, let’s create a “blueprint” using the Patch file.
In order to compile the Patch file properly with Heavy, we can only use supported PD objects, otherwise there will be compilation errors. PD objects not supported by Heavy will be listed in Part 2.
To expose parameters to the target platform (that would be Wwise in this case), we need to declare them in the following manner:
- In PD, press Ctrl+1 to create an object, then type “r SampleRateDivisionL @hv_param 4 24000 24000”.
- The "r" indicates that the object receives information from outside PD.
- SampleRateDivisionL will display parameter names in the plug-in UI.
- "@hv_param" indicates that the object will be compiled into an interface parameter by Heavy.
- 4, 24000 and 24000 are the minimum, maximum and default parameter values.
r indicates that the object receives parameter values from the outside
(mandatory) @hv-param indicates that the input/output values are parameters, @hv_event indicates that they are events
SampleRateDivisionL will show parameter names in the plug-in UI within Wwise
1. In order to process the input signal independently on the left and right channels and increase the asymmetry, I’m going to create a parameter for each channel.
- phasor~ object is the sawtooth oscillator used as a sample clock;
- samphold~ object is the S/H module itself;
- dac~ object is the target where the audio signal will be output.
Note: In the Patch, adc~ represents the input signal. If the source Patch contains an adc~ object, it will generate an effect plug-in; if not, it will generate a source plug-in.
2. Let’s see how the Patch code works.
Note: I added an hslider object to control the sample rate. It will be useless after you generate the plug-in, so please remember to remove it.
3. Save the Patch file to J:\HeavyAudioPlayground\bitcrusher.pd.
We've now created a “blueprint”. I will show you how to build an “automatic workshop” with the Heavy compiler, and then use it to generate a plug-in and deploy it in our game, in Part 2 of this blog. Stay tuned!
This blog has been translated from Chinese to English.
October 11, 2019 at 11:29 am
Amazing work! Does it mean you'll show us how to deploy in Unity and Android app bulds? That would be very useful !