Hello, I'm trying to integrate Wwise into my project, and I'm currently dealing with Vivox voice chat. I need to stream its audio into Wwise.
I followed this documentation page (
https://www.audiokinetic.com/en/public-library/2024.1.7_8863/?source=UE4&id=using_features_audioinput.html) to get started, but I'm failing to get it working.
My code is here
https://gist.github.com/Tonetfal/52b2898e346715ad507c7ee7f4690ad9. For some reason forum is telling me that the message is over 12000 characters, so I can't paste it here.
As you can see, I'm dealing with shorts here, so int16, unlike the documentation page, which is using IVoiceCapture::GetVoiceData(), which outputs uint8. IncomingRawVoiceData and CollectedRawVoiceData are both TArray<int16>.
Vivox audio buffers are interleaved, I deal with it in the FillSamplesBuffer function, which is very similar to what the documentation page has, but it's simply not working. There's no audio, and it accesses invalid array elements frequently, it depends on the amount of raw voice data that's coming in.
I assume it has to do with the fact that I'm dealing with int16 instead. In fact there's NumBytesPerSample documentation introduces, which is ultimately used to compute the number of available bytes it can read, however, I don't quite understand how it's supposed to work. In the original code it was diving and multiplying some values by it, but I don't understand why it would do that at all, considering that it's dealing with int8, which is 1 byte, not 2.
Another thing I don't understand is the NumSamples Wwise API is giving me. Is it the number of samples per channel or for all the channels?
How should I deal with interleaved buffers in case there's not enough data? Right now, if NumSamples is 512, it'll read values from 0 to 511 for buffer 0, and values from 512 to 1023 for buffer 1. However, if I only have 300 collected samples, it'll correctly read the ones that I have from 0 to 149, but the other ones aren't at index 512+, they are at index 150+.
Can someone clear my doubts and give an example of how I could rewrite it correctly?