The Making of SoundSeed Grain

신규 출시 / 사운드 디자인 / Wwise에 대한 팁과 도구

We have wanted to build a general purpose granular synthesizer for Wwise for a very long time  due to the wide range of sounds it can produce, and the many ways it can be used in games.
I remember having made a granular synthesis-like patch in Native Instruments’ Reaktor for myself, several years ago. I used it a little, and then found that it could not do exactly what I wanted, so I added new features. And then added even more features. And again. And it ended up like this...
Picture1
A horrible wall of buttons (not Reaktor - Reaktor is cool, my design wasn’t)
Because granular synthesis can produce a wide variety of sounds, I think that it is very easy to design a granular synthesizer organically, so that each feature is piled up on top of the others in what ends up being a big giant mess.

Focus versus Flexibility

It is commonly accepted that an audio plug-in is usually more successful when it is efficient at doing one specific thing very well. For example, Crankcase Rev uses granular synthesis, along with very sophisticated signal processing in order to successfully emulate the sound of combustion engines. Their UX and choice of user controls is thought out to make these kinds of sounds easy to create. Obviously, we wouldn’t want to make a plug-in that specializes in combustion engines, as Rev already does it very well.
While it infringes the rule stated above about being focussed, we wanted to create a plug-in that would cover most applications that require granular synthesis, without necessarily being specialized in any of them. Hopefully, a critical mass of users will be able to tweak it so that it works for their applications. So, we started with this simple question: What do people really expect from granular synthesis for their game audio?
Picture2
Mock up of the simplest granular synth
Picture3
A later mock-up.

User Research

While attempting to design a synthesizer that can do everything without being specialized in anything, it became obvious that we needed to get our community of Wwise users involved.
We asked the question: “What would you use a (traditional) granular synthesizer for (in your work)?” Here’s a sample of answers we received:
  • “car engines, textured ambiences, rolling sounds, rain, fire”
  • “explosions, destruction, oneshot, crumbling sounds”
  • “for evolving ambience bed, for sci-fi or ‘semi-natural’"
  • "weird, unnatural sound"
  • “wavetable synth caps for making music...”
  • “implementing run-time / interactive ‘effects’ with lots of control. For example: ‘make the monster hit sound more grainy / scarce as the monster scales are ripped off its skin.’"
  • “generating some cool granular sounds that I'd want to bounce back into a DAW and edit”
Many said that they “did not know yet”, and one person even said that they couldn’t find any use for it!
Certainly, the use cases would span a wide range. So, reaching out to the community did not stop at gathering use cases.
  • We gathered written conversations we already had with designers about granular synthesis before even starting the design.
  • We carried out interviews in-house.
  • We built a quick prototype and brought it with us to GDC, where we hoped to find people interested in trying it.
  • We made a second prototype that we shipped to users and carried out follow-up interviews where they would share their thoughts and ideas with us.
  • We shipped an alpha to get more feedback.
It was a long process of learning, testing, and validating, and we gained valuable knowledge at every step of this process. Questions users responded to, and topics of conversations during interviews we had with them were centred around:
  • Use cases;
  • Workflow and pain points;
  • Specific questions about how features were presented;
  • Missing features.
Conversations would often go way beyond that!
Picture4
The first functional prototype (GDC)
Picture5
The prototype sent out to users.
While I could go into detail about a gazillion things that came out of these interviews, I’ll just mention one example to give you an idea about what this process is like. (Just in case you’d like to be involved next time!)

Example: Grain Window

One user’s ambition was to design a car engine.They explained to us how they expected to see a “window”, on top of the waveform, from which grains would be randomly picked up. Also, these grains should correspond to explosions in the original file. And the window itself should move with an RTPC that is tied to some game-driven RPM.
Picture6
Expressing allowed grain start positions with a window and markers.
A special feature “grain window” for selecting grain start positions would be appropriate for this kind of application but not for others such as time-scale effects and wavetable synths. The same behavior could be equivalently set up by using random modulators on the Position cursor, but it would be difficult to visualize how modulation would translate into such a window and impossible to have grains start at explosions.
Picture7
Alpha
So, while we decided not to add a special “Grain Window” feature, which would be optimal for this car engine use case, this concept and the discussion around it ended up inspiring the features Snap to Markers and automatic transient detection, and Grain Visualization (overlaid on the waveform view). We think these features would cater to a larger variety of use cases. You can see how these features, along with modulation of Position can be used to emulate a “grain window” here.
This is instead how you would set up the synthesizer to emulate a “grain window”:
  1. Assign the Position property to an RTPC, which you will bind to an appropriate game parameter, say, RPM. This RTPC defines the left boundary of the macro window.
  2. Assign the Position property to a Random+ modulator with some Amount. The chosen Amount defines the width of the macro window, in percentage of the file's duration.
  3. If the width of the window should change with the RPM, bind it to RPM using an RTPC.
You may force grains to always start at precise positions in the original sound, for example at each explosion, by adding markers and using the "Snap to markers" option. Automatic transient detection may save you time.
Picture8
2018.1.4, showing in action envelope visualizer, grain window, and live monitoring under sliders.
To conclude, SoundSeed Grain was shaped by continuous valuable user feedback throughout development. Other concrete examples of features we were able to tweak as a result include:
  • The Envelope Visualizer (center of the window), without which users would struggle to understand the interactions between grain duration, emission, and envelope shape.
  • The slider feedback (live monitoring under each slider), necessary to work with modulation.
Picture9
The Envelope Visualizer
 
Picture10
Slider with live feedback
We will certainly pursue this user-oriented approach for other plug-ins, so if YOU would like to be part of the discussion about the next one, by all means, register here!

Acknowledgment

Audiokinetic would like to thank everyone who participated in the design process, our wonderful community who gave us feedback, and, in particular, Tom Bible, Nicholas Bonardi, Mike Caviezel, and Andrew Quinn for their contributions and immense help towards figuring out where we needed to go! 

Before you go, here's a quick video on SoundSeed Grain!  

 

  Subscribe

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni

Director, R&D

Audiokinetic

Louis-Xavier Buffoni leads the research team at Audiokinetic and has been focusing on spatial audio, sound synthesis, audio coding and machine learning.

 @xbuffoni

댓글

댓글 달기

이메일 주소는 공개되지 않습니다.

다른 글

Wwise와 Unity로 12일만에 상호작용 폴리 디자인하기

버튼을 누를 때마다 캐릭터의 외모, 태도, 자세가 끊임없이 새로워지는 캐릭터가 걷는다는 것은 폴리 아티스트에게 정말 꿈만 같으면서도 동시에 악몽이기도 하죠. 꿈이든 악몽이든,...

24.9.2019 - 작성자: 피에르-마리 블랑 (Pierre-Marie Blind)

Wwise를 사용한 음향 시뮬레이션

최근에 작업한 프로젝트에서 저희 회사는 어느 고객의 미래 사무실 공간 음향 시뮬레이션을 Wwise를 사용해 포로토타입할 기회가 있었습니다. 새로운 건물에서 음향이 어떻게 들릴...

2.10.2019 - 작성자: 에길 샌드펠드 (Egil Sandfeld)

게임의 사회성에 큰 역할을 하는 음성 채팅: Wwise GME 서비스 시작!

상호작용 요소가 많은 인기 게임의 사운드는 잘 만들어진 게임 자체의 '개별적인 캐릭터'를 뒷받침해줄 뿐만 아니라 더 풍부한 플레이 경험을 제공하고 게임 시스템을 강화하는 데 큰...

8.1.2020 - 작성자: 시몽 아슈비 (Simon Ashby)

새로운 기능 개요 2019.2

Profiler Filtering (프로파일러 필터링) Capture 세션을 보면, 주어진 시간 안에 생기는 많은 캡처 데이터를 어렵지 않게 발견할 수 있습니다. 이는...

14.2.2020 - 작성자: Audiokinetic

이미지 기반 파라미터를 이용한 오픈 월드 앰비언트 디자인

Blend Container는 강력한 시퀀싱 및 믹싱 도구가 될 수 있습니다. 단순히 그것의 기능을 배우는 것이 게임 사운드 디자이너의 생각에 온갖 종류의 새로운 아이디어를...

13.3.2020 - 작성자: 톰 토디아 (TOM TODIA)

머더러스 퍼수츠(Murderous Pursuits)의 대사와 대화 설계 - 제 2부

안녕하세요, 머더러스 퍼수츠의 대화와 대사 시스템의 제 2부로 다시 돌아온 제이미입니다. 제 1부를 아직 보지 않으셨다면 여기를 클릭하여 읽어주세요! 제 1부에서는 게임에서의...

20.5.2020 - 작성자: 제이미 크로스(JAIME CROSS)

다른 글

Wwise와 Unity로 12일만에 상호작용 폴리 디자인하기

버튼을 누를 때마다 캐릭터의 외모, 태도, 자세가 끊임없이 새로워지는 캐릭터가 걷는다는 것은 폴리 아티스트에게 정말 꿈만 같으면서도 동시에 악몽이기도 하죠. 꿈이든 악몽이든,...

Wwise를 사용한 음향 시뮬레이션

최근에 작업한 프로젝트에서 저희 회사는 어느 고객의 미래 사무실 공간 음향 시뮬레이션을 Wwise를 사용해 포로토타입할 기회가 있었습니다. 새로운 건물에서 음향이 어떻게 들릴...

게임의 사회성에 큰 역할을 하는 음성 채팅: Wwise GME 서비스 시작!

상호작용 요소가 많은 인기 게임의 사운드는 잘 만들어진 게임 자체의 '개별적인 캐릭터'를 뒷받침해줄 뿐만 아니라 더 풍부한 플레이 경험을 제공하고 게임 시스템을 강화하는 데 큰...