Visualizing Ambisonics with the 3D Meter

사운드 디자인 / 공간 음향

Ambisonics

Since version 2016.1, Wwise has supported ambisonic channel formats for busses. Wwise takes care of encoding (mixing a non-ambisonic signal to an ambisonic bus) and decoding (mixing an ambisonic signal to a non-ambisonic bus). This allows users to mix multiple sounds in the environment to an ambisonics bus, and then convert that ambisonics bus to binaural using a 3D audio plug-in such as Auro Headphone. Users can also export the ambisonic audio directly by using the Wwise Recorder Effect on an ambisonics bus.

Displaying Ambisonics

One of the major benefits of the ambisonics channel format is that it is speaker-setup-agnostic. Through the process of decoding, it can be targeted to various types of speaker setups. This also allows the sound field to be easily rotated with the listener. However, this also introduces a challenge in how to display the audio data meaningfully, in a user interface setting. In an ambisonic channel format, each channel relates to a specific 'spherical harmonic' in the sound field. What this means is that the meter data for a channel determines the sound level at multiple points in space. This makes it almost impossible to understand the spatial information conveyed in an ambisonics channel configuration just by observing the channel meters alone. Instead, the encoded ambisonic signal (represented by the channel information) must first be decoded to a specific speaker setup, such that the levels at each speaker in the target setup can be inspected. This kind of technique has been used previously by Kronlachner and Zotter (2014) and Lecomte and Gauthier (2015).

When decoding an ambisonic signal to a target speaker setup, the spatial accuracy of the reconstructed sound field is affected by the distribution of the loudspeakers (Trevino et al., 2010). Specifically, the extent to which each direction is equally sampled by the speaker setup can simplify (or complicate) the decoding process. In order to display meaningful spatial information from an ambisonic channel configuration we must first decode the data to a specific virtual speaker setup, and then use that virtual speaker setup to display the spatialized data. This is the method used by the 3D Meter in order to display the spatialized ambisonic data.

The 3D Meter

The 3D Meter provides a way of visualizing the directional information in an ambisonics bus. It presents the audio data as a spherical heatmap, by decoding the ambisonic channel data onto virtual speaker positions around a sphere in the local bus space (as mentioned above). These virtual speakers are then individually metered, and the levels at each speaker are used to construct the smoothed heatmap. The speaker levels are sent to a custom pixel shader, which is used to render a sphere. Each pixel in the sphere is colored according to its proximity to each virtual speaker position around the surface of the sphere and the levels at each position. The 3D Meter, therefore, allows users to inspect the directional information contained in an ambisonics bus and how it is affected by the sources distributed around the listener. There are four individual panels in the 3D Meter that can each be used to view the sound field from a different perspective. Six fixed orthographic viewpoints are available (top & bottom, front & back, left & right), as well as the perspective viewpoint, which can be rotated around the sound field by clicking & dragging. All viewpoints can also be zoomed using the mouse wheel.

 

Media_1

The 3D Meter also provides a good way of comparing the spatial resolution of different ambisonics channel formats. The examples below compare the spatial resolution of first-order, second-order, and third-order ambisonics busses. In each example, the same two sources are mixed into the bus; one panned to front center, and one panned to rear center. It can be seen that third-order ambisonics provides more accurate spatial localization of the individual sources, whereas with lower-order ambisonics formats they become blended together. In the examples below, the viewpoints are set up as follows: top-left panel—back; top-right panelfront; bottom-left paneltop; bottom-right panelperspective.

 

 

Media_2

Front and back sources mixed into first-order ambisonics bus

 

Media_3

Front and back sources mixed into second-order ambisonics bus

 

Media_4

Front and back sources mixed into third-order ambisonics bus

 

Color Mapping

The colors of the heatmap are used to indicate different dB ranges, and these mappings are configurable in the Meter Settings window. In the settings window, the dB threshold for a color indicates the beginning of the gradient between that color and the next. So, in the example below, the meter will blend between black and blue over the range of -20 dB to -16 dB. It will then blend between blue and green over the range of -16 dB to -12 dB, and so on.

 

MEdia_5

In order to exclude a color from the meter, its dB threshold can be set to lower than any of the thresholds preceding it. In the below example, green is excluded from the color map displayed on the meter.

MEdia_6

Smoothing

The 3D Meter implements smoothing when metering the virtual speaker positions around the sphere. This smoothing works in the same way as the Meter plug-in Effect. The attack value determines the time it takes for the display value to rise by 10 dB and the release value determines the time it takes for the display value to fall by 10 dB. Note that if a Meter plug-in Effect is used on a single source, before it is mixed into an ambisonics bus, the meter values on the Meter Effect for the source will not match those displayed on the 3D Meter for the ambisonics bus. This is due to gain factors that are applied during the ambisonics decoding process. When the original source is metered it is metered directly, without any spatialization. When it is mixed to an ambisonic signal, the source is spatialized and 'moved' to some point a fixed distance away from the listener. Therefore, when the ambisonic signal is decoded and metered, the original source's level within the ambisonic sound field will be lower than when metered individually. The example below features a choppy helicopter sound moving around the listener position in a circle. As the release value is decreased, the meter reacts to changes in level more quickly, causing more rapid change in display. As the release value is increased, the meter values are smoothed out.

Media_7

References
  • Kronlachner, M. and Zotter, F., 2014, February. Spatial transformations for the enhancement of Ambisonic recordings. In Proceedings of the 2nd International Conference on Spatial Audio, Erlangen.
  • Lecomte, P. and Gauthier, P.A., 2015, November. Real-time 3D ambisonics using FAUST, processing, pure data, and OSC. In 15th International Conference on Digital Audio Effects (DAFx-15), Trondheim.
  • Trevino, J., Okamoto, T., Iwaya, Y. and Suzuki, Y., 2010, August. High order Ambisonic decoding method for irregular loudspeaker arrays. In Proceedings of 20th International Congress on Acoustics (pp. 23-27).

Sean Soraghan

Software Developer

Audiokinetic

Sean Soraghan

Software Developer

Audiokinetic

Sean Soraghan is a Software Developer at Audiokinetic working on the Authoring tool as well as the Unity and Unreal game engine integrations. He has previously completed an engineering doctorate, researching the representation and visualization of musical timbre, and has collaborated on large-scale audio-visual installations and exhibitions. He enjoys developing games and tools in his spare time.

댓글

댓글 달기

이메일 주소는 공개되지 않습니다.

다른 글

Wwise를 사용한 음향 시뮬레이션

최근에 작업한 프로젝트에서 저희 회사는 어느 고객의 미래 사무실 공간 음향 시뮬레이션을 Wwise를 사용해 포로토타입할 기회가 있었습니다. 새로운 건물에서 음향이 어떻게 들릴...

2.10.2019 - 작성자: 에길 샌드펠드 (Egil Sandfeld)

이미지 기반 파라미터를 이용한 오픈 월드 앰비언트 디자인

Blend Container는 강력한 시퀀싱 및 믹싱 도구가 될 수 있습니다. 단순히 그것의 기능을 배우는 것이 게임 사운드 디자이너의 생각에 온갖 종류의 새로운 아이디어를...

13.3.2020 - 작성자: 톰 토디아 (TOM TODIA)

머더러스 퍼수츠(Murderous Pursuits)의 대사와 대화 설계 - 제 2부

안녕하세요, 머더러스 퍼수츠의 대화와 대사 시스템의 제 2부로 다시 돌아온 제이미입니다. 제 1부를 아직 보지 않으셨다면 여기를 클릭하여 읽어주세요! 제 1부에서는 게임에서의...

20.5.2020 - 작성자: 제이미 크로스(JAIME CROSS)

Wwise State-based Mixing의 새로운 기능 - 온갖 파라미터들!

2017.1 이전에는 States 탭에서는 볼륨, 피치, 저역-통과 필터, 하이패스 필터 및 메이크업 게인과 같은 비교적 적은 수의 파라미터에만 액세스할 수...

9.6.2020 - 작성자: Bradley Meyer

소규모 게임 프로젝트가 Wwise로부터 혜택을 받을 수 있는 5가지 이유

여러분이 게임 오디오 분야에 종사하고 있으며 이전에 소규모 게임 프로젝트를 수행한 적이 있는 경우. 다음과 같은 대화를 나눈 적이 있을 수 있습니다. "근데, 와이즈와 같은...

7.7.2020 - 작성자: 알렉스 메이 (ALEX MAY)

상호작용 사운드를 통한 기능적 오디오 인터페이스: 자동차 안전, 안내 및 엔터테인먼트 응용을 위한 청각적 디스플레이 사용하기

본 백서에서는, 자동차 HMI(Human-Machine Interface)를 향상시키고 전반적인 사용자 환경을 개선하기 위해 소리가 혁신적인 방법으로 사용되는 고급 사용 사례를...

15.7.2020 - 작성자: 프랑수아 티볼트 (François Thibault)

다른 글

Wwise를 사용한 음향 시뮬레이션

최근에 작업한 프로젝트에서 저희 회사는 어느 고객의 미래 사무실 공간 음향 시뮬레이션을 Wwise를 사용해 포로토타입할 기회가 있었습니다. 새로운 건물에서 음향이 어떻게 들릴...

이미지 기반 파라미터를 이용한 오픈 월드 앰비언트 디자인

Blend Container는 강력한 시퀀싱 및 믹싱 도구가 될 수 있습니다. 단순히 그것의 기능을 배우는 것이 게임 사운드 디자이너의 생각에 온갖 종류의 새로운 아이디어를...

머더러스 퍼수츠(Murderous Pursuits)의 대사와 대화 설계 - 제 2부

안녕하세요, 머더러스 퍼수츠의 대화와 대사 시스템의 제 2부로 다시 돌아온 제이미입니다. 제 1부를 아직 보지 않으셨다면 여기를 클릭하여 읽어주세요! 제 1부에서는 게임에서의...