4

Trying to understand the Web Audio API better. We're using it to create an AudioContext and then sending audio to be transcribed. I want to be able to determine when there is a natural pause in speech or when the user stopped speaking.

Is there some data in onaudioprocess callback that can be accessed to determine pauses/breaks in speech?

let context = new AudioContext();
context.onstatechange = () => {};

this.setState({ context: context });

let source = context.createMediaStreamSource(stream);
let processor = context.createScriptProcessor(4096, 1, 1);
source.connect(processor);
processor.connect(context.destination);
processor.onaudioprocess = (event) => {
   // Do some magic here
}

I tried a solution that is suggested on this post but did not achieve the results I need. Post: HTML Audio recording until silence?

When I parse for silence as the post suggests, I get the same result - either 0 or 128

let context = new AudioContext();

let source = context.createMediaStreamSource(stream);
let processor = context.createScriptProcessor(4096, 1, 1);
source.connect(processor);
processor.connect(context.destination);

/***
   * Crete analyser
   *
   **/
let analyser = context.createAnalyser();
analyser.smoothingTimeConstant = 0;
analyser.fftSize = 2048;
let buffLength = analyser.frequencyBinCount;
let arrayFreqDomain = new Uint8Array(buffLength);
let arrayTimeDomain = new Uint8Array(buffLength);
processor.connect(analyser);

processor.onaudioprocess = (event) => {
   /**
   *
   * Parse live real-time buffer looking for silence
   *
   **/
   let f, t;
   analyser.getByteFrequencyData(arrayFreqDomain);
   analyser.getByteTimeDomainData(arrayTimeDomain);
   for (var i = 0; i < buffLength; i++) {
       arrayFreqDomain[i]; <---- gives 0 value always
       arrayTimeDomain[i]; <---- gives 128 value always
   }
}

Looking at the documentation for the getByteFrequencyData method I can see how it is supposed to be giving a different value (in the documentation example it will give a different barHeight), but it isn't working for me. https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode/getByteFrequencyData#Example

islalobo
  • 617
  • 2
  • 8
  • 22
  • 1
    Possible duplicate of [HTML Audio recording until silence?](https://stackoverflow.com/questions/24515978/html-audio-recording-until-silence) – Patrick Feb 13 '19 at 15:10
  • It's not a duplicate because I tried implementing the suggested solution on that post without achieving the results I need. – islalobo Feb 13 '19 at 15:59
  • look into using "clip" which avg of vol over period of a second ... https://webrtc.github.io/samples/src/content/getusermedia/volume/ – Robert Rowntree Feb 13 '19 at 16:35
  • This demo doesn't work for me in Chrome 72.0.3626.96. Looking at it in Firefox I get a bit more response, but I think I would want to use instant. That said, when I tried to copy some of the code from the demo source code I get an error `SoundMeter is not defined` – islalobo Feb 13 '19 at 17:02
  • Not really a duplicate **question**, but [this answer](https://stackoverflow.com/questions/46543341/how-can-i-extract-the-preceding-audio-from-microphone-as-a-buffer-when-silence/46781986#46781986) may help you. You don't need a ScriptProcessor here, only an AudioAnalyser. – Kaiido Mar 20 '19 at 08:58

0 Answers0