Spatial Audio Signal Flow Confusion

Question

Using Wwise 2023.1.7.

I'm heavily confused with the signal flow of spacial audio. I've watched the Wwise On Airs, looked through the documentation (especially: Obstruction and Occlusion and Voice Pipeline) and tried out a lot, but I seem to be too stupid... I hope, that someone can shed some light onto this.

Just to be clear:
When I speak about spatial audio, I mean all the spatial attenuation features of Wwise like transmission, diffraction, obstruction and occlusion.
With 3D spatialization I mean the enabling of listener relative routing and selecting position and orientation in 3D spatialization.
With 3D audio I mean the object based audio device mode, that uses the spatial audio capabilities of each platform. (This i don't want to use to be able to control binauralization fully myself)

My main question is: When enable transmission and diffraction is enabled. Is each found path an additional audio object? Using something like multi position behind the scenes? Or is it simply calculating the lowest/highest values of volume, hpf and lpf and applies this to the single object at the end (like the voice profiler would let you believe)? Or is it maybe different depending on if 3D spatialization on the source is ticked or maybe it's again different when using 3D audio? Or maybe this is just a panning matter like described here? Both possibilities lead to a set of other questions:

One signal path:

• Does it just take the virtual position of the nearest diffraction path?
• What happens if there is diffraction and transmission? The audio from transmission should be straight from the source, the diffraction from the virtual position.

Multiple "parallel processing" signals

• How is avoided that the volume jumps if a diffraction path is added and the transmission isn't fully blocking?
• Is there a way to inject custom dsp into some of these paths as we'd like to have a custom filter for diffraction with a different slope.

I have more questions, but it probably doesn't make sense to fully think both cases through, if only one is happening.

Furthermore. It's a bit confusing that enabling the volume and/or filter for transmission has a continuous falloff effect based on the virtual source distance when moving an obstacle in between, but it makes sense. However when I'm connected to Unity and I obstruct a source by 100%, when moving the lpf curve up and down in wwise at 100 percent it makes a sine like swell instead of going from full lpf filtering to none. I documented this isolated here(yt), where everything is turned off except transmission lpf. Transmission and Diffraction is of course also enabled. Maybe this is a bug?

What I also see when using a binaural spatialization effect (Steam Audio), that the virtual source position isn't taken at all for spatialization - only for the attenuation curves. So disabling 3D spatialization disables not only panning but also some things connected to those virtual sources? Maybe that's a flaw in the steam audio spatializer effect and could be better programmed, maybe by making it an object processor plugin?

Sorry, that's quite a lot to read through. Thanks in advance for any answer even if it's just to only one part.

Answer 1

Hello Jonas,

You ask good questions.

To answer your main question: it's a bit of both. More precisely, multiple found paths are internally treated as "propagation paths," which are functionally equivalent to multipositions. You can see these paths in the Voice Inspector's List View by expanding the Attenuation row (refer to the screenshots in this documentation).

However, for performance reasons, as shown in the Voice Profiler or the Voice Graph pane of the Voice Inspector, all these paths/multipositions are ultimately collapsed into a single signal that is filtered only once. The resulting volume, LPF, HPF, and panning matrix are calculated using a complex algorithm that attempts to account for the perceptual loudness of each path when computing their respective contributions. The panning of the resulting signal is more or less a weighted average of all path directions. Of course, this is inherently an approximation. And unfortunately, it's not possible to apply custom DSP to these paths individually.

Note that when using object-based panning in Wwise, each “multiposition” results in a new object being spawned.

How is it avoided that the volume jumps when a diffraction path is added and the transmission isn’t fully blocking?

Typically, if transmission loss is 0%, you should avoid having diffraction in addition. In other words, why have a portal if the wall is effectively transparent? If the transmission loss is greater, the difference in volume compared to a direct line of sight should help compensate for the addition of the diffraction path. In theory, a sudden volume change could occur when diffraction is introduced, but this is mitigated by the path merging behavior I mentioned earlier: two paths coming from the same direction will never be louder than one. That’s one of the benefits of the current calculation method.

I'm not entirely sure I understand the issue you're encountering in Unity. It’s possible that it's a side effect of the path merging calculations.

As for the Steam Audio binauralizer, I’m not familiar with it specifically. But yes, it should ideally be implemented as an Object Processor. If it's a mixer plugin, it may not support multiposition, which Wwise Spatial Audio relies on quite heavily.