Web Speech API " Detecting known words"

Question

I'm in the process of developing an app which can detect certain key words. I have been using googles web speech.

I'm able to record the users voice and have it print out, but I want to be able to detect certain words.

I can't seem to find a way of doing this. I've found this question Detecting known words using the Web Speech API but the site in the answer it links to, is gone.

Can anyone shed a light on this? any help would be great. Cheers! here is a fiddle

  var recognition = new webkitSpeechRecognition(); // google voice recognition start

  recognition.continuous = true; //so that recognition will continue even if the user pauses while speaking


  document.getElementById('startSpeech').addEventListener('click', function() {
   startSpeech();
 });
  document.getElementById('stopRecognizing').addEventListener('click', function() {
      stopRecognizing();
  });


  var startSpeech = function(){

  var lang = ['en-GB', 'United Kingdom'];
  final_transcript = '';
    recognition.lang =  recognition.lang
    recognition.start() //activates the speech recognizer
  }

  var stopRecognizing = function(){
   recognition.stop();//stops the mic from listening
   return;
  }  


 recognition.onresult = function(event) {
 console.log('event', event);
 // This handler concatenates all
 // the results received so far into two strings  final_transcript & interim_transcript

var interim_transcript = '';

for (var i = event.resultIndex; i < event.results.length; ++i) {
  if (event.results[i].isFinal) {
    final_transcript += event.results[i][0].transcript;
  } else {
    interim_transcript += event.results[i][0].transcript;
  }
}
//final_transcript = capitalize(final_transcript);
    final_span.innerHTML = linebreak(final_transcript);
    interim_span.innerHTML = linebreak(interim_transcript);

    // console.log('final_transcript' , typeof final_transcript);
    // console.log('interim_transcript' , interim_transcript);

    if (final_transcript.length > 0) {
        matchingKeyWords(final_transcript);
    }
};

      function linebreak(s) {
    var two_line = /\n\n/g;
    var one_line = /\n/g;
    return s.replace(two_line, '<p></p>').replace(one_line, '<br>');
}

function capitalize(s) {
    var first_char = /\S/;
    return s.replace(first_char, function(m) { return m.toUpperCase(); });
}


        var matchingKeyWords = function(conversation){

    var keyWords = ['power', 'tie', 'imaginary' ,'bomb', 'drink', 'underwear', 'chance', 'type', 'historical',
        'society' ,'collect' ,'sugar']

        for (var i = 0; i < keyWords.length; ++i) {
            console.log(conversation[0]);
            if (conversation === keyWords[1]) {
                console.log('hit');
            }
            // console.log('keyWords[i]', keyWords[i]);
        }
}

score 2 · Answer 1 · answered Jul 05 '16 at 16:28

You categorically cannot modify the API's in any way to append dictionaries or influence the results.

HOWEVER you can boil down the results to words which are phonetically similar to influence the interpretation by your code.

At the moment you appear to be performing 'exact' matches for certain words. So if the engine thinks it hears "Thai" instead of "Tie"... that outcome would not yield a match.

The solution is to add 'fuzzyness' to the matching logic. Have you considered a library like FuzzySet.js?

You can find plenty more examples by searching for something like "fuzzy string matching JS" or "phonetically similar JS"

This is a great replacement for the faulty grammers bug as noted above. Thanks! — Julian Dormon, Mar 07 '21 at 21:28

score 1 · Answer 2 · answered Oct 03 '16 at 14:01

1

In the matchingKeyWords function inside for loop Please Change keyWords[1] to keyWords[i] then try. Also change the if condition to

if(conversation.indexof(keyWords[i]) > -1)

answered Oct 03 '16 at 14:01

Imran Mohammed

73
1
1
5

score 1 · Answer 3 · answered Mar 10 '17 at 05:06

I see that this question is eight months old, and you've probably found the answer already, but in case anyone else is still looking – I think you need to give the app some "grammar" to check against, like this:

var colors = [ 'aqua' , 'azure' , 'beige', 'bisque', 'black', 'blue', 'brown', 'chocolate', 'coral' ... ];
var grammar = '#JSGF V1.0; grammar colors; public <color> = ' + colors.join(' | ') + ' ;'

MDN have a demo for Web Speech API that is similar to what you're trying to do (if I understand correctly). You read a phrase aloud and it tells you whether it's understood.

http://mdn.github.io/web-speech-api/phrase-matcher/

There is a Chromium bug that makes using grammars useless. https://stackoverflow.com/questions/41373579/the-effect-of-the-grammar-in-the-web-speech-api — Julian Dormon, Mar 07 '21 at 21:28

Web Speech API " Detecting known words"

3 Answers3