3

I am trying to capture audio from the default device and stream with the help of OPUS.

The audio from the device is captured using WASAPI.

Encoding and decoding is done using OPUS where the states are created initially.

Initialization logic

//SAMPLE_RATE: 48000, CHANNELS: 2, BITRATE: 64000

//encoder
encoder = opus_encoder_create(SAMPLE_RATE, CHANNELS, APPLICATION, &err);
if (err < 0)
{
    fprintf(stderr, "failed to create an encoder: %s\n", opus_strerror(err));
    return err;
}

err = opus_encoder_ctl(encoder, OPUS_SET_BITRATE(BITRATE));
if (err < 0)
{
   fprintf(stderr, "failed to set bitrate: %s\n", opus_strerror(err));
   return err;
}

//Decoder
decoder = opus_decoder_create(SAMPLE_RATE, CHANNELS, &err);
if (err < 0)
{
   fprintf(stderr, "failed to create an encoder: %s\n", opus_strerror(err));
   return err;
}

The captured audio is stored as byte array and later used for encoding. I am doing the little endian conversion for the byte array since the encoder expects the input array in __int16 format.

Encoder logic

//SAMPLE RATE: 48KHz, FRAME_SIZE : 960 , CHANNELS : 2 & MAX_PACKET_SIZE : 3828 

unsigned char cbits[MAX_PACKET_SIZE] = "";
opus_int16 in[CHANNELS * FRAME_SIZE] = { 0 };
BYTE* data = new (std::nothrow) BYTE[NoOfFrames * _FrameSize];

for (i = 0; i < CHANNELS * NoOfFrames; i++)
{
    in[i] = data[2 * i + 1] << 8 | data[2 * i];
}

nbBytes = opus_encode(encoderState, in, FRAME_SIZE, cbits, MAX_PACKET_SIZE);
if (nbBytes < 0)
{
    fprintf(stderr, "encode failed: %s\n", opus_strerror(nbBytes));
    return;
}

The byte stream obtained from the encoder is later passed to decoder. The output data obtained is in __int16 format, so it is restored back using little endian format. This pcm data is stored in the form of wav file in order to verify the encoder and decoder process.

Decoder logic

#CHANNELS: 2

opus_int16 out[MAX_FRAME_SIZE * CHANNELS] = { 0 };
unsigned char pcm_data[MAX_FRAME_SIZE * CHANNELS * sizeof(opus_int16)] = "";

#assume encodedData and encodedDataBytes are obtained after encoding

nDecodedSamples = opus_decode(decoderState, encodedData, encodedDataBytes, out, MAX_FRAME_SIZE, 0);
if (nDecodedSamples < 0)
{
   fprintf(stderr, "decode failed: %s\n", opus_strerror(nDecodedSamples));
   return;
}

for (i = 0; i < CHANNELS * nDecodedSamples; i++)
{
    pcm_data[2 * i] = out[i] & 0xFF;
    pcm_data[2 * i + 1] = (out[i] >> 8) & 0xFF;
}

Difficulties faced:

The decoded output stored in wav format contains much noise. Also tried to import the raw pcm data using Audacity application, same noise is generated. Suspecting issue in params used for both encoder and decoder methods, correct me if wrongly used.

Capture audio from device reference: https://github.com/microsoft/Windows-classic-samples/blob/main/Samples/Win7Samples/multimedia/audio/CaptureSharedEventDriven/WASAPICapture.cpp

Encoder and decoder reference: https://chromium.googlesource.com/chromium/deps/opus/+/1.1.1/doc/trivial_example.c

Any ideas are welcomed.

Preethi Ravi
  • 47
  • 3
  • https://stackoverflow.com/questions/33589354/audiounit-opus-codec-crackle-issue – Hans Passant Oct 03 '22 at 08:32
  • How do you store it in WAV format? Did you try to store plain pcm like in the "trivial_example.c" ? – TeaAge Solutions Oct 05 '22 at 07:32
  • You can store the pcm byte data along with wav headers. [Example code](https://github.com/microsoft/Windows-classic-samples/blob/27ffb0811ca761741502feaefdb591aebf592193/Samples/Win7Samples/multimedia/audio/CaptureSharedTimerDriven/WASAPICaptureSharedTimerDriven.cpp#L377) – Preethi Ravi Oct 06 '22 at 13:20

0 Answers0