Please note that we will be performing routine server maintenance on November 23rd. Your experience may be temporarily limited on the Audiokinetic website and Wwise Launcher for a period of 48 hours.
Blog homepage

Working with Object-Based Audio

Sound Design / Spatial Audio / Wwise Tips & Tools

Object-Based Sound Systems

The market's desire for immersive audio experiences has propelled an array of speaker configurations that project far beyond 7.1, encompassing the height dimension alongside an enhanced planar capacity. This has forced some surround sound systems such as Dolby Atmos and DTS:X, as well as emerging distribution standards like MPEG-H, to come up with the concept of audio objects in order to abstract the speaker configuration onto which spatialized audio content should be rendered.

A very important distinction between channel-based and object-based audio is that channel-based relies on a fixed number of channels from the point of production to playback, and audio mixes are designed for a strict speaker configuration or have to be downmixed to accommodate simpler speaker set-ups. Object-based audio implies that ’audio objects’ (i.e. mono source + 3D coordinates relative to the listener) must be preserved, and passed over to the renderer to spatialize over the user set-up. In other words, object-based audio holds ‘objects’ or audio elements as individual streams and allows each stream to contain metadata describing where it should play in a three dimensional space, agnostic to the speaker set-up. Audio objects can be individually placed spatially with precision during production, and each audio object is rendered in real time to the right combination of speakers through a process called adaptive rendering, which compliments speaker layouts. Adaptive rendering also permits adjustments to the mix on the receiving end.

The object paradigm translates well with binaural virtualization of audio, common with VR. This process is typically implemented by taking each individual audio object, and passing it through a filter (HRTF). Its frequency response will depend on the 3D position of the source relative to the listener.

 

The Drawbacks of Preserving Objects

While object-based audio looks at optimizing the quality of the rendering from a spatial perspective, preserving objects comes with limitations on the audio design process because it is incompatible with important audio authoring practices.

For example, sound designers may wish to control the dynamic range of a group of sounds. They typically do so by adding a compressor on a bus which sums the inputs – in other words, creates a submix. Submixing is used for various other reasons such as:

  • Performance
  • Side-chaining
  • Mastering
  • Metering, loudness metering
  • Grouping

In Wwise, objects of the Master-Mixer Hierarchy are used for these purposes. A bus becomes a mixing bus when it needs to. 

Now, say you route two 3D sources to a mixing bus, the audio data that you get at the output of the bus will represent a superposition of multiple sources/objects, from which you will have lost the constituting directional components.

Screen_Shot_2016-06-17_at_2.34.19_PM.png

Figure 1 – Two sources with different incident angles are panned and mixed onto a multichannel bus. The action of panning implies converting the spatial representation of the input (here, expressed with an angle of incidence Ɵi) to that of the output (for example, by distributing the signal on two or more of a 7.1 speaker setup). The output is a mixture of panned objects, and does not have a distinct angle of incidence (it is not an object, in the sense that was given earlier).

 

Therefore, object-based audio is incompatible with submixes because objects only exist upstream of mixing, and this can limit the sound designer from using authoring practices that are habitually applied to submixes.

 

Case Study

The mixer plug-in interface in Wwise is an object-based plug-in architecture, because mixer plug-ins require the use of individual sound sources or ‘objects’ in order to access the 3D position of each input and produce binauralized audio. While they appear to be a great framework for implementing binaural virtualizers, using them will in fact impose some constraints on the design process due to the limitations of object-based audio:

1. You should not add Effects on a binaural signal (at least not non-linear Effects such as compressors) because they can break the spatial audio illusion you are trying to achieve with binaural processing.

Screen_Shot_2016-06-17_at_2.46.12_PM.png

Figure 2 – Incorrectly adding a compressor Effect downstream on the binaural signal at the output of the mixer plug-in.

 

2. You should not use mixing busses upstream of the mixer plug-in, because the resulting submix will no longer contain the 3D information needed for binaural processing.

Screen_Shot_2016-06-22_at_5.30.15_PM.pngFigure 3 – Mixing objects upstream of the mixer plug-in. The submixed signal does not carry a single object and, therefore, cannot be binauralized by the plug-in.

 

 

The trade-off

Now, wouldn’t it be nice if we could have the best of both worlds? If we could render audio with decent spatial accuracy without the shortcomings of object-based audio? Stay tuned to learn about how ambisonics may help you out with this!

Screen_Shot_2016-06-17_at_2.47.53_PM.png

Figure 4 – Objects are submixed freely and panned onto a multichannel spatial audio representation, which conveys the directional intensity of all its constituents and can also be manipulated by adding, for example, Effects.

 

  

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Xavier Buffoni leads spatial audio research at Audiokinetic. He holds a Master’s degree in Electrical Engineering and occasionally writes articles and gives lectures on various audio topics such as interactive music, dynamix mixing with HDR, and more recently, object-based panning and ambisonics. Xavier has presented talks at AES for Games, and MIGS, amongst other industry events.

 @xbuffoni

Comments

Leave a Reply

Your email address will not be published.

More articles

Hellion - Sounds from Outer Space

Hellion is a first person space survival game that is entirely based on the laws of Newtonian...

16.5.2017 - By Marko Zivanovic

Planet Coaster - Crowd Management : Using Data to Generate Dynamic Crowd Audio (PART 1)

Part 1. Scaling Ambition Part 2. The Crowd Soundbox System Part 3. Additional Layers Planet...

27.6.2017 - By Planet Coaster

Working With the New 3D-Bus Architecture in Wwise 2017.1: Simulating an Audio Surveillance System

Imagine you are developing a “covert espionage” game that features an audio surveillance system – a...

4.7.2017 - By Nathan Harris

Aporia: Beyond the Valley - Making Sounds for a Ghost

What does a ghostly, dark, mysterious, and dangerous entity sound like? And how does it affect its...

29.8.2017 - By Troels Nygaard

Fun with Feedback

Introduced with Wwise 2017.1, 3D busses and auxiliary sends from busses make it possible to use the...

17.10.2017 - By Nathan Harris

A Day Without Sound

As a lifelong music lover and someone who has worked with sound for nearly twenty years, it occurred...

12.12.2017 - By Dr. Karen Collins

More articles

Hellion - Sounds from Outer Space

Hellion is a first person space survival game that is entirely based on the laws of Newtonian...

Planet Coaster - Crowd Management : Using Data to Generate Dynamic Crowd Audio (PART 1)

Part 1. Scaling Ambition Part 2. The Crowd Soundbox System Part 3. Additional Layers Planet...

Working With the New 3D-Bus Architecture in Wwise 2017.1: Simulating an Audio Surveillance System

Imagine you are developing a “covert espionage” game that features an audio surveillance system – a...