Using Ambisonics for Dynamic Ambiences

사운드 디자인 / 공간 음향 / Wwise에 대한 팁과 도구

Rotating Ambisonics

As we have seen in Ambisonics as an Intermediate Spatial Representation (for VR), ambisonics is a multichannel audio format that can favorably represent the spatiality of an audio mix. An ambisonic audio file/mix/signal can thus be regarded as a sound field, a mixture of different sound sources coming from different directions and arriving at a listener's ears with different angles of incidence. 

Rotation by matrixing

One useful property of the ambisonic representation is that it can be rotated very easily, by cleverly manipulating the matrix of mixing gains. In terms of resources (CPU and memory), this is equivalent to calculating panning gains; and, because this is done at every audio frame (~10-20 ms) instead of at every audio sample (~2 µs), it is orders of magnitude less demanding than filtering or even mixing (adding) audio signals.

After having rotated an ambisonic signal, you get a new ambisonic signal where the apparent angles of incidence of all its constituents have been rotated accordingly. 

Ambisonics rotation in Wwise

Wwise implements rotation of ambisonic sound fields automatically. The rotation corresponds to the relative orientation of the associated game object and its listener. To enable this magic, you simply need to set the ambisonic sound positioning to 3D Game-Defined and route it to an ambisonic bus, as depicted in the screenshots below. The game object and listener orientations are driven by the game through the Wwise API.

Figure 1 - Positioning and routing required for enabling rotation of ambisonics in Wwise. Notice the 3D and Game-Defined options selected in (a).

(Fig 1-a)

 

Figure 1 - Positioning and routing required for enabling rotation of ambisonics in Wwise. (b) shows how to set a bus to ambisonics.

(Fig 1-b)

Figure 1 - Positioning and routing required for enabling rotation of ambisonics in Wwise.  The routing can be observed in the Profiler view (c).

(Fig 1-c)

 

Figure 1 - Positioning and routing required for enabling rotation of ambisonics in Wwise. The actual rotation is given by the relative game object and listener orientations driven by the game engine, here in Unity (d).

(Fig1-d)

Figure 1 - Positioning and routing required for enabling rotation of ambisonics in Wwise. Notice the 3D and Game-Defined options selected in (a). The asset in question is routed to a bus with an ambisonic channel configuration. (b) shows how to set a bus to ambisonics. The routing can be observed in the Profiler view (c). The actual rotation is given by the relative game object and listener orientations driven by the game engine, here in Unity (d). 

The following audio example illustrates ambisonic rotation. You should listen to it with headphones. Audio sample 1 was produced by playing an ambisonic file and converting it to binaural using the Auro Headphone plug-in in Wwise. In order to produce Audio sample 2, we simply changed the orientation of the listener (using a game engine) to point to the left, while the sound field's object orientation points to the front. The listener is rotated counterclockwise (towards the left), and this is why you hear the sound field rotated clockwise.

Audio sample 1 

Audio sample 2

 

Ambiences Using Ambisonics Assets

The ease with which the channels of an ambisonics signal/bed/stream can be manipulated in order to rotate the sound field it represents makes it great for implementing dynamic ambiances in games. You need to play them back on game objects that are constant in orientation and point towards a specific reference. For example, if there is an auditory element with clear directionality in a recording, which maps to a visual element in the game, you will want to orient the game objects so that they align. When the Listener navigates in the space and changes orientation, the ambisonics sound field is automatically rotated as it is being mixed into the bus. Consequently, the sound field is coherent with the visuals.

Using recorded material

In Wwise, you can import ambisonics recordings that you have captured using an appropriate coincident microphone (such as http://www.core-sound.com/TetraMic/TetraMic-small3.jpg) after converting them to B-format (conventional 1st-order ambisonics). Most microphones come with their own software to transform the raw microphone signal, typically in A-format, to B-format**.

Here is an example of rotation done in Wwise with a field recording. The original audio, Audio sample 3 has been recorded using an SPS-200 microphone, converted to B-Format, and imported in Wwise. The most noticeable auditory event is the car horn, located slightly to the left in the original sample. To produce Audio sample 4, we turned the listener 90 degrees towards the left (as we did in the previous example), our next step was to play back and record the output. The car horn moved mostly to the right and all other sound sources rotated accordingly.

 

Audio sample 3

Audio sample 4

 

"Bouncing" Ambiences

It is possible to generate ambisonic sound files from an interactive session in Wwise using the Wwise Recorder plug-in. For example, you may create a complex ambience in the Wwise authoring tool by using a plurality of mono sources placed in space, mixing them in an ambisonics signal, and then recording that ambisonics signal using the Wwise Recorder plug-in. The next step would be to reimport the recorded ambisonics asset into your Wwise project, and use it as a rotatable 3D ambience in your game, exactly as if it were recorded material. 

This workflow is similar to the "bouncing" or "freezing" option typically available in linear DAWs, such as Nuendo, Reaper, and ProTools. One motivation for bouncing an ambience to disk instead of keeping all its individual components for simultaneous playback is to save runtime resources (CPU and memory). This can be achieved by saving disk space and playing one multichannel file instead of many mono files at the same time.

Bouncing ambiences in Wwise

  1. Create a bus and set its channel configuration to Ambisonics (any order).
  2. Insert a Wwise Recorder plug-in on that bus. Since it is an ambisonics bus, the audio will be written to file in the ambisonics format. 
  3. Route all your individual mono sources (or their common parent) to this bus.
  4. Place each source in space using the 3D User-Defined Position Editor.
  5. Trigger them at opportune times using a Soundcaster Session, an Event, a Random Container, or any combination thereof. When all sounds stop playing, because they are finished or because they are stopped, the audio will be "bounced" to disk. You may import it in your project directly by clicking the Import in Wwise... button on the Recorder plug-in.
  6. Inspect this new asset and set its positioning type to 3D. When played from the game, it will be rotated as described previously.

 

5-bounce-profiler-and-structure.png

(Fig 2-a)

 

6-user-defined_positioning.png

(Fig 2-b)

 

7-Recorder.png

(Fig 2-c)

Figure 2 - The Ambisonics City Ambience Blend Container of the Wwise Sample Project being recorded into a single ambisonics file. The Project Explorer view on the left of screenshot (Fig 2-a) shows the ambience made of the recording mentioned above (Ex1 - Ambisonic City 3D Ambiance), superimposed with a large number of individual sounds, and mixed and played back using various behaviors specified in the Random Containers. Screenshot (Fig 2-b) shows an example of how an asset (or group of assets) has been placed in space using the 3D User-Defined Position Editor. Screenshot (Fig 2-c) shows the settings of the Wwise Recorder plug-in. More details on this plug-in

 

As explained in a previous blog, Ambisonics as an Intermediate Spatial Representation (for VR), you may record up to 3rd order ambisonics for improved spatial precision, but, the increase in the required number of channels (9 channels for 2nd order and 16 for 3rd order) and, consequently, memory demands, may outweigh such improvements. Also, while recorded ambisonics retains the relative arrival direction of its individual components, it does not allow modifying their individual properties–such as their relative volume and filtering–afterwards***.  

 

* As long as the ambisonics format is "full-sphere".

** Wwise accepts AMB files in FuMa convention, identified by a proper GUID in their headers. See more

*** Some signal processing techniques allow one to do such things. However, in our context they are fairly irrelevant since we already have access to the individual components prior to mixing them.  

Figure 2 - The Ambisonics City Ambience Blend Container of the Wwise Sample Project being recorded into a single ambisonics file. The Project Explorer view on the left of screenshot (Fig 2-a) shows the ambience made of the recording mentioned above (Ex1 - Ambisonic City 3D Ambiance), superimposed with a large number of individual sounds, and mixed and played back using various behaviors specified in the Random Containers. Screenshot (Fig 2-b) shows an example of how an asset (or group of assets) has been placed in space using the 3D User-Defined Position Editor. Screenshot (Fig 2-c) shows the settings of the Wwise Recorder plug-in. More details on this plug-in. 

-Screenshot Figure 1(d): Unity 3D 

-Recorded 3D Ambience Sample, and modified version: 

Ambisonic B-Format ambiance from A-Format SPS200 microphone recording.
Audiokinetic Inc. All rights reserved.
Audio content provided by Studio Lab SONORUM
(c) 2016. www.sonorum.ca

The Audio Content contained in this file is the property of Studio Lab SONORUM. Your sole right with regard to this Audio Content is to listen to the Audio Content as part of this Sample Project. This Audio Content cannot be used or modified for a commercial purpose nor for public demonstration.

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni leads the research team at Audiokinetic and has been focusing on spatial audio, sound synthesis, audio coding and machine learning.

 @xbuffoni

댓글

David Eisler

September 20, 2016 at 02:52 pm

Wow. This seems really groundbreaking, or at least awesome

댓글 달기

이메일 주소는 공개되지 않습니다.

다른 글

Wwise 2019.1 출시!

Wwise 2019.1이 출시되었습니다. Wwise 런처에서 새로운 Wwise를 다운로드하세요. Wwise 2019.1 버전에는 GDC 2019에서 가장 뜨거운 관심을 받았던...

2.4.2019 - 작성자: Audiokinetic (오디오키네틱)

Wwise CPU 최적화: 기본 가이드라인

오디오 제작자에게 힘을 실어준다는 것은, 한편으론 이들의 손에 일부 게임 리소스의 책임을 넘긴다는 것을 뜻합니다. Wwise는 편집기와 SDK를 통해 최소한의 CPU 예산을 지킬...

29.5.2019 - 작성자: 아드리앙 라보아 (Adrien Lavoie)

ABZÛ - 도전! 물고기 10,000 마리의 게임 오디오 만들기

이 이미지를 가장 처음 봤을 때 여러분은 어떤 생각이 먼저 드나요? 저는 어떻게 하면 이 월드에 현실적이고 진짜 같은 사운드를 만들 수 있는지, 그리고 이 장면의 중심으로 헤엄을...

30.7.2019 - 작성자: 스티브 그린 (Steve Green)

이미지 기반 파라미터를 이용한 오픈 월드 앰비언트 디자인

Blend Container는 강력한 시퀀싱 및 믹싱 도구가 될 수 있습니다. 단순히 그것의 기능을 배우는 것이 게임 사운드 디자이너의 생각에 온갖 종류의 새로운 아이디어를...

13.3.2020 - 작성자: 톰 토디아 (TOM TODIA)

Wwise를 사용하여 보다 복잡한 MIDI로 제어되는 샘플 악기만들기

MIDI 기반 음악으로 되돌아가는 데에 관심이 있는 사람들은, PS4의 내장 신디사이저를 이용할 수 없으며(존재하지 않음) PC 사운드카드의 GENERAL MIDI에 의존할 수도...

28.7.2020 - 작성자: 다니엘 벡 (DANIEL BECK)

합성 만으로 빗소리 만들기

몇 년 전 저는 원하는 모든 사운드를 합성할 수 있을까 하는 궁금증이 생겼습니다. 바람, 새의 노랫소리, 곤충 소리 등 다양한 자연 소리를 합성하기 시작했죠. 이런 작업에서는...

3.9.2020 - 작성자: 알렉산더 킬코(ALEKSANDR KHILKO)

다른 글

Wwise 2019.1 출시!

Wwise 2019.1이 출시되었습니다. Wwise 런처에서 새로운 Wwise를 다운로드하세요. Wwise 2019.1 버전에는 GDC 2019에서 가장 뜨거운 관심을 받았던...

Wwise CPU 최적화: 기본 가이드라인

오디오 제작자에게 힘을 실어준다는 것은, 한편으론 이들의 손에 일부 게임 리소스의 책임을 넘긴다는 것을 뜻합니다. Wwise는 편집기와 SDK를 통해 최소한의 CPU 예산을 지킬...

ABZÛ - 도전! 물고기 10,000 마리의 게임 오디오 만들기

이 이미지를 가장 처음 봤을 때 여러분은 어떤 생각이 먼저 드나요? 저는 어떻게 하면 이 월드에 현실적이고 진짜 같은 사운드를 만들 수 있는지, 그리고 이 장면의 중심으로 헤엄을...