How to record voice on Android Things

Question

I am working on Preview 2 of the SDK of Android Things on RPi3. Tried Audio Recorder as well as Media Recorder, still not able to get audio captured. I am trying to get my speech converted to text. Regular SpeechRecognition is not supported. I have USB MIC connected directly into RPi3 as well as Headphone MIC connected to RPi3 via USB sound card.

MediaRecorder Code:

private void startRecording() {
    Log.d(TAG, "startRecording....");
    mRecorder = new MediaRecorder();
    Log.d(TAG, "startRecording: Audio Source"+MediaRecorder.getAudioSourceMax());
    mRecorder.setAudioSource(MediaRecorder.AudioSource.UNPROCESSED);
    mRecorder.setOutputFormat(MediaRecorder.OutputFormat.RAW_AMR);
    mRecorder.setOutputFile(mFileName);
    mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.DEFAULT);

    try {
        mRecorder.prepare();
    } catch (IOException e) {
        Log.e(TAG, "prepare() failed");
    }

    mRecorder.start();
}

private void stopRecording() {
    // stops the recording activity
    if (mRecorder != null) {
        mRecorder.stop();
        mRecorder.release();
        mRecorder = null;
    }
}

Manifest Permission:

 <uses-permission android:name="android.permission.RECORD_AUDIO" />

Error:

03-09 17:17:38.662  3970  3970 D MainActivity: onCreate
03-09 17:17:38.668  3970  3970 D MainActivity: startRecording....
03-09 17:17:38.672  3970  3970 D MainActivity: startRecording: Audio Source9
03-09 17:17:38.678   161   161 E AudioSystem: AudioSystem::getInputBufferSize failed sampleRate 8000 format 0x1 channelMask 10
03-09 17:17:38.678   161   161 E AudioRecord: AudioSystem could not query the input buffer size for sampleRate 8000, format 0x1, channelMask 0x10; status -22
03-09 17:17:38.678   161   161 E StagefrightRecorder: audio source is not initialized
03-09 17:17:38.678  3970  3970 E MediaRecorder: start failed: -2147483648
03-09 17:17:38.680  3970  3970 D AndroidRuntime: Shutting down VM
03-09 17:17:38.683  3970  3970 E AndroidRuntime: FATAL EXCEPTION: main
03-09 17:17:38.683  3970  3970 E AndroidRuntime: Process: com.example.androidthings.myproject, PID: 3970
03-09 17:17:38.683  3970  3970 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.androidthings.myproject/com.example.androidthings.myproject.MainActivity}: java.lang.RuntimeException: start failed.
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2646)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2707)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.ActivityThread.-wrap12(ActivityThread.java)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1460)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.os.Handler.dispatchMessage(Handler.java:102)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.os.Looper.loop(Looper.java:154)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.ActivityThread.main(ActivityThread.java:6077)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at java.lang.reflect.Method.invoke(Native Method)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755)
03-09 17:17:38.683  3970  3970 E AndroidRuntime: Caused by: java.lang.RuntimeException: start failed.
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.media.MediaRecorder.start(Native Method)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at com.example.androidthings.myproject.MainActivity.startRecording(MainActivity.java:181)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at com.example.androidthings.myproject.MainActivity.onCreate(MainActivity.java:63)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.Activity.performCreate(Activity.java:6662)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1118)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2599)
03-09 17:17:38.683  3970  3970 E AndroidRuntime:        ... 9 more

score 2 · Answer 1 · answered Mar 09 '17 at 23:08

I'll start with the easy part. Since speech recognition services are not available in the current preview, you might want to look at alternative methods for handling speech to text in your app. Here is a blog post that can point you to some options.

Regarding audio record, here are a couple thoughts. You might have better luck using AudioSource.MIC in your code instead. I haven't tested MediaRecorder directly, but another suggestion would be to use AudioRecord directly. Especially since your goal is to pass the audio data off to another service for processing (and probably not just to save it to a file). This will give you the ability to work with each sampled audio buffer. Below is an example of something that should work on an Android Things device to initialize audio recording:

// Audio recording parameters
private static final int SAMPLE_RATE = 44100;
private static final int ENCODING_FORMAT = AudioFormat.ENCODING_PCM_16BIT;
private static final int CHANNEL_FORMAT = AudioFormat.CHANNEL_IN_MONO;

private AudioRecord mRecorder;
private final int mBufferSize = AudioRecord
        .getMinBufferSize(SAMPLE_RATE, CHANNEL_FORMAT, ENCODING_FORMAT);

public void initAudioRecorder() {
    if (mRecorder == null) {
        try {
            mRecorder = new AudioRecord.Builder()
                    .setAudioSource(MediaRecorder.AudioSource.MIC)
                    .setAudioFormat(new AudioFormat.Builder()
                            .setEncoding(ENCODING_FORMAT)
                            .setSampleRate(SAMPLE_RATE)
                            .setChannelMask(CHANNEL_FORMAT)
                            .build())
                    .setBufferSizeInBytes(2*mBufferSize)
                    .build();
            mRecorder.startRecording();
        } catch (UnsupportedOperationException e) {
            Log.w(TAG, "Unable to initialize recording", e);
        }
    }
}

The above code I have tried earlier and it didn't worked. However the blog is exactly what I want. But either with inbuilt speech API (case #1) or with google cloud speech api (case #2). Case #1 I have tried earlier but I was missing the base apk in my device. For Case #2 I need to capture audio first which is where I am stuck !! — Abhinav Tyagi, Mar 10 '17 at 17:15

score 0 · Answer 2 · answered Mar 09 '17 at 17:28

Each device may have different initialization settings,, so you will have to create a method that loops over all possible combinations of bit rates, encoding... :

private static int[] mSampleRates = new int[] { 8000, 11025, 22050, 44100 };
public AudioRecord findAudioRecord() {
    for (int rate : mSampleRates) {
        for (short audioFormat : new short[] { AudioFormat.ENCODING_PCM_8BIT, AudioFormat.ENCODING_PCM_16BIT }) {
            for (short channelConfig : new short[] { AudioFormat.CHANNEL_IN_MONO, AudioFormat.CHANNEL_IN_STEREO }) {
                try {
                    Log.d(C.TAG, "Attempting rate " + rate + "Hz, bits: " + audioFormat + ", channel: "
                            + channelConfig);
                    int bufferSize = AudioRecord.getMinBufferSize(rate, channelConfig, audioFormat);

                    if (bufferSize != AudioRecord.ERROR_BAD_VALUE) {
                        // check if we can instantiate and have a success
                        AudioRecord recorder = new AudioRecord(AudioSource.DEFAULT, rate, channelConfig, audioFormat, bufferSize);

                        if (recorder.getState() == AudioRecord.STATE_INITIALIZED)
                            return recorder;
                    }
                } catch (Exception e) {
                    Log.e(C.TAG, rate + "Exception, keep trying.",e);
                }
            }
        }
    }
    return null;
}

AudioRecord recorder = findAudioRecord();
recorder.release();

this gave a bad buffer size for all the combination resulting in NULL AudioRecorder — Abhinav Tyagi, Mar 10 '17 at 17:12

score 0 · Answer 3 · answered Mar 15 '17 at 22:00

I've gotten speech recognition working with RPi 3 / Things Preview 2 / USB mic / Kõnele (https://github.com/Kaljurand/K6nele), in other words: Things Preview 2 does support audio recording and SpeechRecognition (I assume you mean SpeechRecognizer).

Kõnele recognizes Estonian out of the box. If you want some other language then you need to change the server URL in the Kõnele preferences (or override it with the "ee.ioc.phon.android.extra.SERVER_URL" EXTRA) and set up a recognition server at this URL. The easiest way to start the server is described here: https://github.com/jcsilva/docker-kaldi-gstreamer-server

How to record voice on Android Things

3 Answers3