3

I'm using the Android oboe library for high performance audio in a music game.

In the assets folder I have 2 .raw files (both 48000Hz 16 bit PCM wavs and about 60kB) std_kit_sn.raw std_kit_ht.raw

These are loaded into memory as SoundRecordings and added to a Mixer. kSampleRateHz is 48000:

stdSN= SoundRecording::loadFromAssets(mAssetManager, "std_kit_sn.raw");
stdHT= SoundRecording::loadFromAssets(mAssetManager, "std_kit_ht.raw");
mMixer.addTrack(stdSN);
mMixer.addTrack(stdFT);

// Create a builder
AudioStreamBuilder builder;
builder.setFormat(AudioFormat::I16);
builder.setChannelCount(1);
builder.setSampleRate(kSampleRateHz);
builder.setCallback(this);
builder.setPerformanceMode(PerformanceMode::LowLatency);
builder.setSharingMode(SharingMode::Exclusive);
LOGD("After creating a builder");

// Open stream
Result result = builder.openStream(&mAudioStream);
if (result != Result::OK){
    LOGE("Failed to open stream. Error: %s", convertToText(result));
}
LOGD("After openstream");

// Reduce stream latency by setting the buffer size to a multiple of the burst size
mAudioStream->setBufferSizeInFrames(mAudioStream->getFramesPerBurst() * 2);

// Start the stream
result = mAudioStream->requestStart();
if (result != Result::OK){
    LOGE("Failed to start stream. Error: %s", convertToText(result));
}
LOGD("After starting stream");

They are called appropriately to play with standard code (as per Google tutorials) at required times:

stdSN->setPlaying(true);
stdHT->setPlaying(true); //Nasty Sound

The audio callback is standard (as per Google tutorials):

DataCallbackResult SoundFunctions::onAudioReady(AudioStream *mAudioStream, void *audioData, int32_t numFrames) {

    // Play the stream
    mMixer.renderAudio(static_cast<int16_t*>(audioData), numFrames);
    return DataCallbackResult::Continue;

}

The std_kit_sn.raw plays fine. But std_kit_ht.raw has a nasty distortion. Both play with low latency. Why is one playing fine and the other has a nasty distortion?

TomV
  • 1,157
  • 1
  • 13
  • 25
  • How does it sound if you play just std_kit_ht.raw by itself? Does it only sound bad when played at the same time as the other sample? If so then maybe the mixer is overflowing. It should scale the multiple tracks, mix them into a float or int32 buffer, then clip before writing to the int16_t audioData array. – philburk Dec 07 '18 at 17:06
  • Yes, it sounds bad when play std_kit_ht.raw by itself and at the same time as the other sample. If I load it up into a sound software, like Audacity, it sounds fine. If I play through pure Android Soundpool, it sounds fine. There's something within the android c++ that's causing std_kit_ht.raw to play with some nasty distortion. – TomV Dec 07 '18 at 17:40
  • There must be some difference between those samples. Are you loading the raw files into Audacity to check them? Or playing the originals. Is the amplitude of one higher than the other? Could it be an Endianness issue. They should saved as LittleEndian for ARM or x86. – philburk Dec 09 '18 at 22:37
  • The std_kit_ht.raw resembles a high tom drum sound. The std_kit_sn.raw resembles a snare drum sound. Both samples have been treated identically. The samples were originally in wav form, so to convert to .raw I used Audacity export file type (Other uncompressed file), header (raw header-less), encoding (signed 16-bit PCM). There is no option to save in LittleEndian for ARM or x86. To test in Audacity, I import Raw data, with encoding signed 16-bit pcm, little endian byte order. – TomV Dec 10 '18 at 11:06
  • I managed to minimise the nasty distortion noise of std_kit_ht.raw by reducing the file size from 60kB to 20kB. Strangely, std_kit_sn.raw plays well as expected as the file size of 65kB. – TomV Dec 12 '18 at 15:05
  • The evidence points to a problem in your source audio files, might be worth posting the full project to github, including those files so others can easily reproduce the issue. – donturner Dec 17 '18 at 17:41
  • Good idea. Here's the project: https://github.com/ThomasVeitch/DrumLoopsAndMetronomePro When you download and run it in Android Studio, you will be able to check out the std_kit_ht.raw specifically by Menu->Settings->Fill Sound->Change to High Tom. Then on Menu->Play if you tap on the animated green/yellow circle on bottom right, you will notice that this sound is not right. – TomV Dec 21 '18 at 21:41

1 Answers1

1

I loaded your sample project and I believe the distortion you hear is caused by clipping/wraparound during mixing of sounds.

The Mixer object from the sample is a summing mixer. It just adds the values of each track together and outputs the sum.

You need to add some code to reduce the volume of each track to avoid exceeding the limits of an int16_t (although you're welcome to file a bug on the oboe project and I'll try to add this in an upcoming version). If you exceed this limit you'll get wraparound which is causing the distortion.

Additionally, your app is hardcoded to run at 22050 frames/sec. This will result in sub-optimal latency across most mobile devices because the stream is forced to upsample to the audio device's native frame rate. A better approach would be to leave the sample rate undefined when opening the stream - this will give you the optimal frame rate for the current audio device - then use a resampler on your source files to supply audio at this frame rate.

donturner
  • 17,867
  • 8
  • 59
  • 81
  • Perhaps I'm naive, but how do you set the volume of each track in the oboe API. Couldn't find it. – TomV Dec 30 '18 at 13:52
  • You wouldn't use the Oboe API for this, you'd just need to update the `Mixer` class (which is part of the sample code, not the core API). To do this you'd just keep track of the volume for each track (using a float between 0 and 1) and when it's time to render that track you multiply each sample value by the track volume. – donturner Dec 30 '18 at 20:53
  • void Mixer::renderAudio(int16_t *audioData, int32_t numFrames) { void Mixer::addTrack(RenderableAudio *renderer){ Am I missing something. The variables and methods in the Mixer class don't seem to hold volume information? – TomV Dec 31 '18 at 14:45
  • 1
    There is no concept of volume in Oboe. If you want to have volume you'll have to change (scale) the value of the audio data in the callback based on you volume variable. – Steven Haggerty Sep 02 '19 at 14:07