MediaDevices.getUserMedia() How can I set audio constraints (sampling rate/bit depth)?

Question

With browser Web API, I'd like to set MediaDevices.getUserMedia constraints attributes, suitable to record audio speech (voice messages), e.g. setting these parameters:

mono
16bit
16KHz

Here my code:

   const mediaStreamConstraints = {
       audio: {
         channelCount: 1,
         sampleRate: 16000,
         sampleSize: 16,
         volume: 1
       },

       video: false
   }

   navigator.mediaDevices.getUserMedia(mediaStreamConstraints)
     .catch( err => serverlog(`ERROR mediaDevices.getUserMedia: ${err}`) )
     .then( stream => {

        // audio recorded as Blob 
        // and the binary data are sent via socketio to a nodejs server
        // that store blob as a file (e.g. audio/inp/audiofile.webm)

      } )

The recorded clip is grabbed and stored (using MediaRecorder API), eventually sent to a nodejs server where the blob is saved as a file and processed (the application is a voicebot).

Something goes wrong and the WebM saved file hasn't the required parameters:

$ mediainfo audio/inp/audiofile.webm
General
Complete name                            : audio/inp/audiofile.webm
Format                                   : WebM
Format version                           : Version 4 / Version 2
File size                                : 2.04 KiB
Writing application                      : Chrome
Writing library                          : Chrome
IsTruncated                              : Yes

Audio
ID                                       : 1
Format                                   : Opus
Codec ID                                 : A_OPUS
Channel(s)                               : 1 channel
Channel positions                        : Front: C
Sampling rate                            : 48.0 kHz
Bit depth                                : 32 bits
Compression mode                         : Lossy
Language                                 : English
Default                                  : Yes
Forced                                   : No

E.g.

Sampling rate                            : 48.0 kHz
Bit depth                                : 32 bits

But constraints would imply different values:

Sampling rate                            : 16 kHz
Bit depth                                : 16 bits

Also the blob, played with anew Audio(audioUrl(blob)).play(), doesn't play. Weird. But all works if constraints are just:

const mediaStreamConstraints = { audio: true }

I checked the browser console and I didn't see any error of navigator.mediaDevices.getUserMedia(mediaStreamConstraints) API call.

BTW, I followed guidelines here:

Note that my user agent is: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 (I'm using a last Brave browser version).

Seems to me that any audio constraints settings not allowed by the browser:

brokes the audio blob,
without raising an error exception (I catched both navigator.mediaDevices.getUserMedia() and new MediaRecorder(...). Isn't this last at least a bug?

My question is:

There is any way to have sampling rate/bit depth as requested?

Or the audio format is "hardcoded"/decided by browser implementation?

BTW, The reason of audio parameters formats is because I want to minimize audio blob size to minimize bandwidth in websocket communications between the browser client and the server, optimizing audio blob exchanges for speech (voice messages)

score 5 · Answer 1 · answered Jul 29 '20 at 16:50

Check the capabilities of your browser first:

let stream = await navigator.mediaDevices.getUserMedia({audio: true});
let track = stream.getAudioTracks()[0];
console.log(track.getCapabilities());

demo output:

autoGainControl: (2) [true, false]
channelCount: {max: 2, min: 1}
deviceId: "default"
echoCancellation: (2) [true, false]
groupId: "1e76386ad54f9ad3548f6f6c14c08e7eff6753f9362d93d8620cc48f546604f5"
latency: {max: 0.01, min: 0.01}
noiseSuppression: (2) [true, false]
sampleRate: {max: 48000, min: 48000}
sampleSize: {max: 16, min: 16}

score 1 · Answer 2 · answered Jun 22 '20 at 17:14

Try setting your audio constraints on the audio media track within your stream before you instantiate MediaRecorder.

Something like this, not debugged:

const constraints = {
   audio: {
     channelCount: 1,
     sampleRate: 16000,
     sampleSize: 16,
     volume: 1
   },

navigator.mediaDevices.getUserMedia({audio:true})
 .catch( err => serverlog(`ERROR mediaDevices.getUserMedia: ${err}`) )
 .then( stream => {

    const audioTracks = stream.getAudioTracks()
    if (audioTracks.length !== 1) throw new Error ('too many tracks???')
    const audioTrack = audioTracks[0]       
    audioTrack.applyConstraints (constraints)
    .then (()=> {

         const mediaRecorder = new MediaRecorder(stream)
         /* etc etc etc */
      } )
    .catch(console.error) /* you might get constraint failure here. */
  } )

All that being said, the Opus audio codec does a good job compressing voice to a reasonable size. Just because it sez 48kHz x 32bits doesn't mean it uses that much bandwidth; the audio signal is compressed.

And, try it on the most recent releases of Google Chrome and/or Firefox. This media stuff is in active development.

Thanks for your help! So you proposal is to set constraints at audioTrack level, on getUserMedia stream. Ok, I'll try asap and I'll feedback. But why do you think this will solve? You had success this way? BTW, Don't you agree that constraints setting failure as I shown without exceptions is a WebAPI bug? — Giorgio Robino, Jun 23 '20 at 10:15
@GiorgioRobino - regarding no exceptions being thrown - I think their use of the word "constraint" is a bit misleading. My understanding is that these are more like "requests". The system will try to satisfy them, but it may not be able to. However, if you specify the "constraint" using "exact" e.g.`sampleRate: {exact: 16000}` then it will generate an error if it can't set the sampling rate to 16000. — Chuck Wooters, Jan 08 '22 at 17:34

score 0 · Answer 3 · answered Mar 09 '21 at 17:20

The answer by O.Jones is on the right track but can be more concise by setting the constraint directly when grabbing the userMedia object.

const constraints = {
    audio: {
        channelCount: 1,
        sampleRate: 16000,
        sampleSize: 16,
        volume: 1
    }
}

navigator.mediaDevices.getUserMedia({audio: constraints}) 
    .then( stream => {
        // Do something with the stream...
    })
    .catch( err => serverlog(`ERROR mediaDevices.getUserMedia: ${err}`) )

score 0 · Answer 4 · answered Jun 05 '22 at 20:37

As mentioned in the comments, downsampling from getUserMedia with the exact keyword in the constraints object fails with an OverconstrainedError:

navigator.mediaDevices.getUserMedia({
  audio: {
    sampleRate: { exact: 16000 }
  },
  video: false
})
.then(stream => {
  // Do something with the stream...
})
.catch(error => { console.log('Error :', error) })

And without the exact keyword, the obtained stream doesn't fulfill the request:

navigator.mediaDevices.getUserMedia({
  audio: {
    sampleRate: 16000
  },
  video: false
})
.then(stream => {
  console.log('Sample rate :', stream.getAudioTracks()[0].getSettings().sampleRate)
  // Shows 'Sample rate : 48000'
})
.catch(error => { console.log('Error :', error) })

You may want to try to downsample the audio stream from the microphone using the WebAudio API and create a new MediaStream to record using the AudioContext.createMediaStreamDestination, documented here.

Here is a code snippet for Chrome and Safari (Firefox throws an exception, see here):

const audioContext = new (window.AudioContext || window.webkitAudioContext)({
    sampleRate: 16000
});

const mediaStream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false })
const mediaStreamSource = audioContext.createMediaStreamSource(mediaStream)
const mediaStreamDestination = audioContext.createMediaStreamDestination()
mediaStreamDestination.channelCount = 1
mediaStreamSource.connect(mediaStreamDestination)

const mediaRecorder = new MediaRecorder(mediaStreamDestination.stream)

MediaDevices.getUserMedia() How can I set audio constraints (sampling rate/bit depth)?

4 Answers4

Linked