Speech recording and Speech to text Android sdk

Question

I am currently trying to record audio the user is saying so that they can play it back at a later time as well as use the Google Speech to Text API, so that I can run analysis on the words the API returns. It seems as though the microphone can either work exclusively with Google Speech to Text, or it can record. I can do both separately, but when I try to run them together on my Android app, the app will stop the Speech to Text and will just start the audio recording. I am running the Audio record in a background thread, while running the speech to text in the main UI thread. It looks something like this right now:

protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    // set up Speech to Text Recognizer 
    mSpeechRecognizer = SpeechRecognizer.createSpeechRecognizer(this);
    mSpeechRecognizer.setRecognitionListener(this);

    //Set up Recording functionality
    bufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, AudioFormat.CHANNEL_IN_MONO,
              AudioFormat.ENCODING_PCM_16BIT);

    buffer = new byte[bufferSize];


    //Set up Record Button
    record = (ToggleButton) findViewById(R.id.record);
    record.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
        @Override
        public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {

            if(isChecked){ //record

                Intent i = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
                i.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,  RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
                i.putExtra(RecognizerIntent.EXTRA_PROMPT, "Speak!");
                i.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, "com.example");
                mSpeechRecognizer.startListening(i);

                // end speech to text   

                recorder = new AudioRecord(MediaRecorder.AudioSource.MIC, SAMPLE_RATE, AudioFormat.CHANNEL_IN_MONO,
                      AudioFormat.ENCODING_PCM_16BIT, 2048);
                recorder.startRecording();
                isRecording = true;
                recordingThread = new Thread(new Runnable() {
                    public void run() {
                        saveAudioDataToFile();
                    }


                }, "AudioRecorder Thread");
                recordingThread.start();

score 3 · Answer 1 · edited May 23 '17 at 12:08

You can not recognize speech and record it with recorder at the same time since the access to microphone is unique. What you can do is to run recognition and get recorded data from the recognizer and process it. For that you can use the method onBufferReceived of the SpeechRecognizer which should give you audio for processing. However, due to the bugs in Google implementation this method is not always called so it is not very reliable. You can check for details

Capturing audio sent to Google's speech recognition server

If you want to capture and recognize simultaneously you might want to consider other speech recognition implementations like CMUSphinx where you can actually get access to recognizer input.

Speech recording and Speech to text Android sdk

1 Answers1