Working With the New 3D-Bus Architecture in Wwise 2017.1: Simulating an Audio Surveillance System

사운드 디자인 / 공간 음향 / Wwise에 대한 팁과 도구

Imagine you are developing a “covert espionage” game that features an audio surveillance system – a gameplay system that allows you to plant a hidden microphone in the world.  The audio is captured at the location of the “bug” or virtual microphone and then transmitted wirelessly to a surveillance van.  Players in the van listen to what is going on in the bugged room, all in real time.  The audio that is captured by the microphone is played through a “virtual loudspeaker” located in the van.  The loudspeaker itself is a spatialized sound emitter in the virtual world – the source of the audio just happens to be a mix of sounds that are occurring at another part of the level.  We will even apply a futz effect to the mix to mimic the lo-fi transmission of the signal and distortion through the loudspeaker.

Wwise 2017.1 introduces the ability to preform 3D spatialization on an audio signal, after mixing together input from any number of 3D sources.  We call this "spatialized sub-mix" a 3D-Bus.  This article will demonstrate how to leverage 3D busses in Wwise 2017.1 to simulate the described behavior in a game.  This scenario was chosen because it highlights a number of new features and changes within Wwise 2017.1, both in the authoring tool and the sound engine.  After reading, you will be able to apply the concepts learned to other audio routing scenarios, including ones suggested at the end of the article.

1-UseCase_3dBus_v2.png

Game Objects

In our example, we will register a minimum of 4 game objects.  In Wwise 2017.1, listeners are also game objects, and game objects can be both listeners and emitters.

  • The player listener: This game object represents the ears of the character listening to the audio in the game.  In our example, this is the character in the van listening to the real-time surveillance.
  • The environmental sound emitter: An object in the game that emits sound.  Perhaps it is an enemy character, or some other sound emitting object of interest to our covert surveillance team.  Realistically, your game would have many objects like these to compose a rich environment, but we will have only one for the sake of simplicity in our example.
  • The microphone: This object represents the placement of the hidden microphone, the “bug”.  The placement of this object is important because only sound emitting game objects that are in range, as defined by their attenuation curves in the Actor-Mixer Hierarchy, will be “picked up” and audible to the microphone.
  • The loudspeaker:The sound emitter game object that emits a spatialized mix of audio captured by the microphone.  The position of the loudspeaker represents the point of retransmission of the sound.  In our example the loudspeaker is in a surveillance van far away from the original source.  If it were, however, close enough to the original source, the player listener would actually be able to hear a mix of the loudspeaker and the original source at the same time.

Listener-Emitter Associations

The listener emitter associations are defined in code by use of the SetListener and SetGameObjectAuxSendValues APIs.  They define the relationship between game objects; that is, which game objects listen to which other game objects.  In Wwise, the only thing that distinguishes an emitter from a listener game object is the listener-emitter associations that have been defined between them using the API.

 In our example, we define the following associations:

  • The environmental sound emitter and the loudspeaker have the player listener as a listener.
  • The microphone has the loudspeaker as a listener.
  • The environmental sound emitter has the microphone as a listener, and has been assigned as such using the SetGameObjectAuxSendValues API.  We set it as an ‘aux send’ listener so that we can use a specific, unique bus from the Wwise project to model the behavior of the microphone in the Wwise Authoring tool.

You can verify that your emitter listener associations have been set up correctly by examining the Emitter-Listener tab in the Advanced Profiler.  Notice that the entry for Environmetal_Sound_Emitter which has Microphone as its listener also indicates it is sending to the Microphone bus and is of routing type “game-defined”.  The routing type "game-defined" indicates that the emitter-listener association and output bus has been established using the game-defined sends API.  Routing type “user-defined”, on the other hand, indicates that the output bus is defined by the Wwise project (in the Master-Mixer Hierarchy, the output bus is the parent bus, and in the Actor-Mixer Hierarchy, the output bus is in the General Settings tab).  If the routing type is "user-defined", the Bus column will be empty because a given game object may have any number of voices and/or buses, each with their own unique output bus as defined by the project (in other words, the user).  The listener game object in the associations with routing type "user-defined" is established by the SetListeners API—there is no way to establish emitter-listener associations in the Wwise project.

2-image2017-5-19_9-44-45.png

 

Setting Up the Wwise Project

 The Actor-Mixer Hierarchy and the Master-Mixer Hierarchy in the Wwise project act as a template for the voice pipeline that is created at run-time. In Wwise 2017.1, a bus instance in the sound engine is always associated with a game object; however, the routing defined in the project is still respected.  We pay special attention to the Enable Positioning check box—this determines if the parent or output bus instance will be spawned on the same game object or on a different game object, as determined by the assigned listeners.

  • First, we create a bus for the loudspeaker, and set it to Enable Positioning.  This will allow this instance of the Loudspeaker bus to have a distinct position in 3D space, defined by the position of the loudspeaker game object.  Furthermore, checking Enable Positioning tells the sound engine to connect the bus to a unique instance of the parent bus for each associated listener.  In our case, the loudspeaker game object has a single listener, the player listener, and the parent bus of the Loudspeaker bus is the Master Audio Bus.  During run-time, an instance of the Loudspeaker bus, when spawned on the loudspeaker game object, will connect to an instance of the Master Audio Bus on the player listener game object.
  • On the Loudspeaker bus, we set the positioning type to 3D.  This tells Wwise to spatialize the output of the Loudspeaker bus, relative to the downstream game object (the player listener).  We create and assign a ‘loudspeaker’ attenuation to define how far the loudspeaker transmits sound. The positioning tab on the Loudspeaker bus should end up looking something like this:
3-image2017-5-19_9-37-19.png
  • For extra street cred, we can add a Wwise Guitar Distortion Effect on the Loudspeaker bus to simulate the transmission loss and distortion introduced by the microphone and the loudspeaker.  The specific settings of the Effect are left as an exercise to the reader, or perhaps, a talented sound designer friend of the reader.

4-image2017-5-19_9-37-46.png

  • To simulate the microphone, we create an Auxiliary Bus that is a child of the Loudspeaker bus.  We use an aux bus so that we can assign it as the target of a game-defined send on all game objects that will be picked up by the microphone, including the previously mentioned environmental sound emitter. The bus hierarchy looks something like this:

5-image2017-5-19_9-37-57.png

  • For the Microphone aux bus, we must also check the Enable Positioning check box; however, we set the positioning to 2D.  We check Enable Positioning because we want the Microphone bus instance to be spawned on a game object (also called microphone) with a position in 3D space that is distinct from the downstream game object, the loudspeaker.  This little Enable Positioning check box is very important!  It tells Wwise that the downstream bus, the Loudspeaker, should be spawned on a different game object (also called loudspeaker).

  • We set positioning type to 2D because we do not want the output of the Microphone bus to be spatialized relative to the loudspeaker.  In this case, we are simulating an electric signal between the microphone and the loudspeaker, rather than physical vibrations—we don’t want the physical distance between and relative positions of the microphone and loudspeaker to affect the panning or attenuation of the sound.  This bus does not use an attenuation.

6-image2017-5-19_9-38-12.png

  • In the Actor-Mixer Hierarchy, we need at least one sound to play that will be picked up by the microphone, and retransmitted by the loudspeaker.  We create an Actor-Mixer object so that all of its children can share the same properties:

7-image2017-5-19_9-38-35.png

  • The only thing to note here is that we must check the ‘Use game-defined auxiliary sends’ box.  This allows us to define routing to the microphone game object and aux bus in game code using the auxiliary sends API.

  • In the positioning tab for the ‘environmental sound’ object, we make sure that Enable Positioning is checked and that positioning type is set to 3D.  This will place the sound in the world using the game object’s position, and ensure that the output is spatialized relative to the listener’s position.  In our example, the Event will be posted on the environmental sound emitter game object, whose listeners are both the microphone via an aux send and the player listener via the output bus.

8-image2017-5-19_9-38-53.png

After setting up Events, building our SoundBanks and posting an Event in the game, we can confirm that our routing is correct by examining the Voice Graph:

9-image2017-5-19_9-41-0-1.png

In Wwise 2017.1, the Voice Graph is a powerful tool for troubleshooting routing issues.  To reinforce associations, game objects are now represented by encompassing boxes that group sound and bus objects together.  We can see in this example that we have a sound ‘Environmental_Sound’ playing on the “Environmental_Sound_Emitter” game object.  We have an aux bus “Microphone” which is playing on a game object by the same name and a bus “Loudspeaker”, also playing on a game object by the same name.  At the end of the chain, we have the “Player_Listener” game object that has a single instance of the Master Audio Bus.

The resulting audio in the game depends on the position of the 4 game objects and the attenuations that have been defined for sounds playing or being mixed on these game objects.  And indeed, the result matches our intuition for the scenario—if the player listener is close to the loudspeaker, they will hear the sound that has been mixed into the microphone; if the player listener is close to the environmental sound emitter, they will hear the sound directly; and finally, the microphone must be close enough to the environmental sound emitter to be able to pick it up.

To conclude, I am going to leave you with a few more possible uses for the new 3D bus architecture in Wwise 2017.1.  I challenge you to apply what you have learned in this article to create these scenarios in your game.

 

Acoustic Portals

You are making a virtual reality game with a heavy emphasis on spatial audio.  It is of upmost importance to simulate acoustic phenomena realistically and convincingly to maintain the illusion.  You would like the listener to be able to hear the reverberated sound coming from an adjacent room.  All sounds in the room will be mixed together, fed into a reverb unit, and then the output of the effect will be spatialized and positioned as if it were coming through a doorway.  The new 3D Bus architecture in Wwise makes this possilble.

 

Early reflections and other effects

Sometimes, applying an Effect per voice is too heavy, but applying it as a send Effect on a shared bus is too broad to be useful.  What you would really like to do is create an instance of a bus for each sound emitter game object on which you can apply your Effect.  Specialization may take place downstream of the sub-mix, or not, if perhaps your plug-in takes care of it.  Wwise Reflect employs this strategy and leverages the 3D bus architecture in Wwise to generate a unique set of early reflections for each sound emitter.  The early reflections are spatialized inside of the Reflect plug-in, relative to the listener’s position. 

 

Clustering sounds to reduce overhead

You have a vehicle sound that is made up of a significant number of component sounds that are driven by a complex RTPC system.  The components originate from different physical locations around the vehicle, so you use individual game object for sound emitters.  However, it is costly to apply Effects and spatialization per individual emitter, so you want to do this only when the listener is suitably close by.  If the listener is far enough away that they won’t notice subtle changes in angle, you want to mix all the sounds together first, before applying processing and spatialization to the entire group of sounds.  The 3D bus architecture in Wwise enables this flexibility.

 

Subscribe

Nathan Harris

Sr. Software Developer, R&D

Audiokinetic

Nathan Harris

Sr. Software Developer, R&D

Audiokinetic

Nathan Harris moved to Montreal in 2012 to pursue his dream, developing cutting edge technology for Audiokinetic which would be deployed in hundreds of games across the globe. Previously a Wwise power user and enthusiast, he now works on the research and technology team as a spatial audio and software architecture specialist. Nathan has a black belt in Brazilian jiu jitsu, and when not on the mats or solving problems for tomorrow's most immersive video games, he enjoys brewing beer and roasting coffee using DIY gadgets.

 @spcmn_spiff

댓글

댓글 달기

이메일 주소는 공개되지 않습니다.

다른 글

Wwise CPU 최적화: 기본 가이드라인

오디오 제작자에게 힘을 실어준다는 것은, 한편으론 이들의 손에 일부 게임 리소스의 책임을 넘긴다는 것을 뜻합니다. Wwise는 편집기와 SDK를 통해 최소한의 CPU 예산을 지킬...

29.5.2019 - 작성자: 아드리앙 라보아 (Adrien Lavoie)

Wwise / Unreal Engine 4 / Unity 3D를 사용해 발자국 소리의 표면 관리하기

사전 제작의 초기 단계에서 사운드 디자이너는 많은 시스템을 프로토타입화해야 하는데, 이를 도와줄 오디오 프로그래머가 없는 경우도 있죠. 다행히도 Wwise는 Unreal...

17.7.2019 - 작성자: 세바스티앙 겔라르 (Sébastien Gaillard)

제 1부 - 니어 : 오토마타(NieR:Automata)의 공간 음향과 Wwise로 구현한 다양한 게임 플레이 유형

니어 : 오토마타는 외계 침략자들이 인류를 달에 추방한 후 황무지가 된 지구에서 펼쳐지는 액션 롤플레잉 게임 (RPG)입니다. 이 게임은 인류가 지구를 되찾기 위해 개발한...

11.9.2019 - 작성자: PlatinumGames Inc. (플래티넘 게임즈)

라우드니스를 처리하는 최상의 방법, 제 1강: 라우드니스 측정 (2부)

이전 블로그인 라우드니스를 처리하는 최상의 방법, 제 1강: 라우드니스 측정 (1부)에서는 지에 양 (디지몽크) 님이 다양한 플랫폼과 콘텐츠 타입에 대한 오디오 표준의 도전 과제와...

20.11.2019 - 작성자: 지에 양 (Jie Yang, 디지몽크)

라우드니스를 처리하는 최상의 방법, 제 2강: 라우드니스와 다이내믹 처리 방법

우리가 여기서 말하는 '라우드니스'는 우리가 예시로 측정하는 도시 소음이나 sound pressure level (SPL) 미터와는 다릅니다. SPL 미터는 소리 압력을 측정해요....

15.1.2020 - 작성자: 지에 양 (Jie Yang, 디지몽크)

새로운 기능 개요 2019.2

Profiler Filtering (프로파일러 필터링) Capture 세션을 보면, 주어진 시간 안에 생기는 많은 캡처 데이터를 어렵지 않게 발견할 수 있습니다. 이는...

14.2.2020 - 작성자: Audiokinetic

다른 글

Wwise CPU 최적화: 기본 가이드라인

오디오 제작자에게 힘을 실어준다는 것은, 한편으론 이들의 손에 일부 게임 리소스의 책임을 넘긴다는 것을 뜻합니다. Wwise는 편집기와 SDK를 통해 최소한의 CPU 예산을 지킬...

Wwise / Unreal Engine 4 / Unity 3D를 사용해 발자국 소리의 표면 관리하기

사전 제작의 초기 단계에서 사운드 디자이너는 많은 시스템을 프로토타입화해야 하는데, 이를 도와줄 오디오 프로그래머가 없는 경우도 있죠. 다행히도 Wwise는 Unreal...

제 1부 - 니어 : 오토마타(NieR:Automata)의 공간 음향과 Wwise로 구현한 다양한 게임 플레이 유형

니어 : 오토마타는 외계 침략자들이 인류를 달에 추방한 후 황무지가 된 지구에서 펼쳐지는 액션 롤플레잉 게임 (RPG)입니다. 이 게임은 인류가 지구를 되찾기 위해 개발한...