Blog homepage

Custom Wwise Listener Position Projection System for Titled 2D View

Audio Programming / Game Audio

Introduction

This is the 2nd of a 3-part tech-blog series by Jater (Ruohao) Xu, sharing the work done for Reverse Collapse: Code Name Bakery. You can read the first article here, where he dives into using Wwise to drive in-game cinematics. Stay tuned for Part 3 in the coming weeks!

Custom Wwise Listener Position Projection System for Titled 2D View

Tech Blog Series | Part 2

Reverse Collapse features a distinctive art style, and one of the artistic choices to make the game stand out is the use of tilted 2D level maps. At first glance, the game presents a 2D top-down camera view. However, behind the scenes, each level of the map is meticulously crafted with a slight tilt from the direction that the camera is facing.

Implementing 3D audio in a typical top-down view is relatively straightforward. However, when dealing with a tilted 2D top-down view like in Reverse Collapse, the default 3D audio system may not provide accurate results. Initially attempting this approach led to issues with attenuation bugs, especially when moving the camera from left to right or top to bottom.

To address this, a customized system is necessary to make adjustments to the default 3D audio system and ensure it works seamlessly with the tilted 2D top-down view.

img1

The image above highlights the issue we're encountering: from the perspective of the camera view frustum, the map appears tilted, leading to inaccuracies in the default audio attenuation.

The solution involves utilizing a simple projection system created with Unity's game objects, scripts, and Wwise RTPC, to establish a projected X, Y, and Z coordinate system specifically for audio purposes. By moving the audio listener from its default position on the main camera to the converted position after the map has been rotated, we can achieve our objectives. This approach seamlessly integrates with the default Wwise audio attenuation. Additionally, we can leverage the customized coordinate system for special cases that require manual adjustment of the attenuation range through Wwise RTPCs.

To achieve this, we first need to set up the projected location, denoted by the magenta cube in the video below, as a prefab game object. This game object should be attachable to any camera with a fixed offset distance to the main camera. It's essential to make this game object a prefab for our project because we have multiple camera setups in the game. Therefore, we need the ability to install or switch the projection system as a child of the game object of any camera setup at runtime. However, it's worth noting that this step may not be the same for every project.

After setting up the prefab game object for the project location, the next step is to add the AkAudioListener.cs script, which comes with our Wwise Unity integration, to the prefab parent game object. This action designates it as the audio listener for the game. Subsequently, we should remove the default audio listener from the MainCamera game object.

img2

The image above shows the prefab setup for our special audio listener game object that is used in the sound projection system; we named it SoundProjectSampler

 

The above video demonstrates the problem of the map from the camera view, and showcases the final working system in the engine.

In Wwise, we configure the RTPC to transfer coordinate data from Unity to Wwise. This allows us to apply the results of the sound modifications we desire. The specific range of each RTPC depends on the unit sizes of the map in each game. In our case, we set the range between 0 to 100.

img3

On the camera object, similarly to the solution above, we attach the script and include these RTPC calls within the LateUpdate()function. While it would also function with Update(), using LateUpdate() ensures that all rendering operations are completed before updating our audio coordinates. This sequencing helps to maintain synchronization between audio and visuals.

AkSoundEngine.SetRTPCValue(CameraDistance_X, Normalization(soundProjectionSampler.transform.position.x, 0.0f, 100.0f));
AkSoundEngine.SetRTPCValue(CameraDistance_Y, Normalization(soundProjectionSampler.transform.position.y, 0.0f, 100.0f));
AkSoundEngine.SetRTPCValue(CameraDistance_Z, Normalization(soundProjectionSampler.transform.position.z, 0.0f, 100.0f));

Note that you may find, in the solution above, we have a helper function called Normalization(), which will convert any given float value based on a minimum and maximum range, to the final range of 0 to 1. It is highly recommended to have something like it to help us unify the range for the RTPC that we are working with. The function can be simply written by following the mathematical normalization formula:

Normalized Value = (Value To Be Normalized - Lower Bound) / (Upper Bound - Lower Bound)

There are plenty of implementation examples of normalization formulas in different programming languages, which include instances in C# and C++; a simple online search will yield many results for you in this case, so we will skip this part here.

With this RTPC hooked up, we can achieve accurate audio results using the default Wwise attenuation system. In addition, we have the flexibility to fine-tune each sound by applying the X, Y, and Z coordinate RTPCs directly to the audio sources themselves. This allows sound designers to set their own attenuation system, or augment it on top of the the default attenuation system.

img4

img5

img6

img7

The example demonstrates how the sound attenuation of the fungus is tuned when they are ignited by grenades in the 3009 gameplay level. Initially, the fungus sound is set up as a 3D sound with Wwise's attenuation, utilizing the default attenuation settings. Additionally, the attenuation is driven by the projected coordinates we created, allowing for further customization and fine-tuning of the sound attenuation based on the precise game map units.

In addition to working with existing attenuation profiles, the X, Y, and Z coordinate RTPCs can also independently create attenuation effects on game object scope sounds. Sound designers can fine-tune the sound on each axis of the map, allowing for more precise adjustments that generate accurate acoustic results in unique map angles, such as those found in Reverse Collapse.

This flexibility empowers sound designers to craft immersive audio experiences that complement the game's visuals and enhance the overall gameplay atmosphere.

Disclaimer: The code snippets utilized in this article are reconstructed generic versions intended solely for illustrative purposes. The underlying logic has been verified to function correctly, specific project-specific API calls and functions have been omitted from the examples due to potential copyright restrictions.

Ruohao (Jater) Xu

Audio Programmer, Technical Sound Designer

Ruohao (Jater) Xu

Audio Programmer, Technical Sound Designer

Jater Xu is a seasoned audio programmer and technical sound designer specializing in interactive audio solutions with Wwise integration in both Unreal and Unity using C++, blueprint, and C#. His work drives the immersive soundscapes in acclaimed games such as Homeworld 3, The Chant, and Reverse Collapse.

Comments

Leave a Reply

Your email address will not be published.

More articles

Why is it important to work with middleware when working on higher budget games?

This article provides a quick overview on some of the benefits in my opinion, of using Wwise, and...

6.11.2018 - By Samy Elia Colombano

Behind the Sound of ‘It Takes Two’ | A Q&A with the Hazelight Audio Team

It Takes Two by Hazelight Studios is a split screen action-adventure platformer co-op game. It...

23.7.2021 - By Hazelight

Loop-Based Car Engine Design with Wwise | Part 1

In this series, we’ll have a look at how to set up a sound design for a simple loop-based car engine...

12.8.2021 - By Arto Koivisto

Solving the Mobile Game Audio Puzzle

This article will try to shed light on how we use Wwise and other tools at Rovio in free-to-play...

19.8.2022 - By Can Uzer

Wwise Licensing and Pricing Philosophy

-Just how does all this stuff work??-It’s been whispered, in the dark recesses of studios around the...

13.4.2023 - By Mike Drummelsmith

Animation Lip Sync with Wwise Meter Plug-in

Introduction This is the last of a 3-part tech-blog series by Jater (Ruohao) Xu, sharing the work...

26.8.2024 - By Ruohao (Jater) Xu

More articles

Why is it important to work with middleware when working on higher budget games?

This article provides a quick overview on some of the benefits in my opinion, of using Wwise, and...

Behind the Sound of ‘It Takes Two’ | A Q&A with the Hazelight Audio Team

It Takes Two by Hazelight Studios is a split screen action-adventure platformer co-op game. It...

Loop-Based Car Engine Design with Wwise | Part 1

In this series, we’ll have a look at how to set up a sound design for a simple loop-based car engine...