Detect 'Whistle' sound in android

Question

I want to detect 'Whistle' sound. For that I have implemented http://code.google.com/p/musicg/

Source code itself having issue. When you start app it is ready for listen but when you go back and again restart detector thread it does not trigger whistle detection.

DetectorThread.java

package weetech.wallpaper.services;

import java.util.LinkedList;

import weetech.wallpaper.utils.Debug;
import android.media.AudioFormat;
import android.media.AudioRecord;

import com.musicg.api.WhistleApi;
import com.musicg.wave.WaveHeader;

public class DetectorThread extends Thread {

private RecorderThread recorder;
private WaveHeader waveHeader;
private WhistleApi whistleApi;
private Thread _thread;

private LinkedList<Boolean> whistleResultList = new LinkedList<Boolean>();
private int numWhistles;
private int totalWhistlesDetected = 0;
private int whistleCheckLength = 3;
private int whistlePassScore = 3;

public DetectorThread(RecorderThread recorder) {
    this.recorder = recorder;
    AudioRecord audioRecord = recorder.getAudioRecord();

    int bitsPerSample = 0;
    if (audioRecord.getAudioFormat() == AudioFormat.ENCODING_PCM_16BIT) {
        bitsPerSample = 16;
    } else if (audioRecord.getAudioFormat() == AudioFormat.ENCODING_PCM_8BIT) {
        bitsPerSample = 8;
    }

    int channel = 0;
    // whistle detection only supports mono channel
    if (audioRecord.getChannelConfiguration() == AudioFormat.CHANNEL_IN_MONO) {
        channel = 1;
    }

    waveHeader = new WaveHeader();
    waveHeader.setChannels(channel);
    waveHeader.setBitsPerSample(bitsPerSample);
    waveHeader.setSampleRate(audioRecord.getSampleRate());
    whistleApi = new WhistleApi(waveHeader);
}

private void initBuffer() {
    numWhistles = 0;
    whistleResultList.clear();

    // init the first frames
    for (int i = 0; i < whistleCheckLength; i++) {
        whistleResultList.add(false);
    }
    // end init the first frames
}

public void start() {
    _thread = new Thread(this);
    _thread.start();
}

public void stopDetection() {
    _thread = null;
}

@Override
public void run() {
    Debug.e("", "DetectorThread started...");

    try {
        byte[] buffer;
        initBuffer();

        Thread thisThread = Thread.currentThread();
        while (_thread == thisThread) {
            // detect sound
            buffer = recorder.getFrameBytes();

            // audio analyst
            if (buffer != null) {
                // sound detected
                // MainActivity.whistleValue = numWhistles;

                // whistle detection
                // System.out.println("*Whistle:");

                try {
                    boolean isWhistle = whistleApi.isWhistle(buffer);
                    Debug.e("", "isWhistle : " + isWhistle + " "
                            + buffer.length);

                    if (whistleResultList.getFirst()) {
                        numWhistles--;
                    }

                    whistleResultList.removeFirst();
                    whistleResultList.add(isWhistle);

                    if (isWhistle) {
                        numWhistles++;
                    }

                    // Debug.e("", "numWhistles : " + numWhistles);

                    if (numWhistles >= whistlePassScore) {
                        // clear buffer
                        initBuffer();
                        totalWhistlesDetected++;

                        Debug.e("", "totalWhistlesDetected : "
                                + totalWhistlesDetected);

                        if (onWhistleListener != null) {
                            onWhistleListener.onWhistle();
                        }
                    }
                } catch (Exception e) {
                    Debug.w("", "" + e.getCause());
                }
                // end whistle detection
            } else {
                // Debug.e("", "no sound detected");
                // no sound detected
                if (whistleResultList.getFirst()) {
                    numWhistles--;
                }
                whistleResultList.removeFirst();
                whistleResultList.add(false);

                // MainActivity.whistleValue = numWhistles;
            }
            // end audio analyst
        }

        Debug.e("", "Terminating detector thread...");

    } catch (Exception e) {
        e.printStackTrace();
    }
}

private OnWhistleListener onWhistleListener;

public void setOnWhistleListener(OnWhistleListener onWhistleListener) {
    this.onWhistleListener = onWhistleListener;
}

public interface OnWhistleListener {
    void onWhistle();
}

public int getTotalWhistlesDetected() {
    return totalWhistlesDetected;
}
}

RecorderThread.java

public class RecorderThread {

private AudioRecord audioRecord;
private int channelConfiguration;
private int audioEncoding;
private int sampleRate;
private int frameByteSize; // for 1024 fft size (16bit sample size)
byte[] buffer;

public RecorderThread() {
    sampleRate = 44100;
    frameByteSize = 1024 * 2;

    channelConfiguration = AudioFormat.CHANNEL_IN_MONO;
    audioEncoding = AudioFormat.ENCODING_PCM_16BIT;

    int recBufSize = AudioRecord.getMinBufferSize(sampleRate,
            channelConfiguration, audioEncoding); // need to be larger than
                                                    // size of a frame
    audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC,
            sampleRate, channelConfiguration, audioEncoding, recBufSize);
    buffer = new byte[frameByteSize];
}

public AudioRecord getAudioRecord() {
    return audioRecord;
}

public boolean isRecording() {
    if (audioRecord.getRecordingState() == AudioRecord.RECORDSTATE_RECORDING) {
        return true;
    }

    return false;
}

public void startRecording() {
    try {
        audioRecord.startRecording();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

public void stopRecording() {
    try {
        audioRecord.stop();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

public byte[] getFrameBytes() {
    audioRecord.read(buffer, 0, frameByteSize);

    // analyze sound
    int totalAbsValue = 0;
    short sample = 0;
    float averageAbsValue = 0.0f;

    for (int i = 0; i < frameByteSize; i += 2) {
        sample = (short) ((buffer[i]) | buffer[i + 1] << 8);
        totalAbsValue += Math.abs(sample);
    }
    averageAbsValue = totalAbsValue / frameByteSize / 2;

    Debug.e("", "averageAbsValue : " + averageAbsValue);

    // no input
    if (averageAbsValue < 30) {
       return null;
    }

    return buffer;
}

}

Usage

   public class DetectionService extends Service implements
    OnWhistleListener {

Handler handler;
private DetectorThread detectorThread;
private RecorderThread recorderThread;

@Override
public void onCreate() {
    super.onCreate();
    handler = new Handler();

}

@Override
public IBinder onBind(Intent intent) {
    return null;
}

@Override
public int onStartCommand(Intent intent, int flags, int startId) {

    try {
        if (intent != null && intent.getExtras() != null) {

            if (intent.getExtras().containsKey("action")) {
                Debug.e("", "action : " + intent.getStringExtra("action"));

                if (intent.getStringExtra("action").equals("start")) {
                    startWhistleDetection();
                }

                if (intent.getStringExtra("action").equals("stop")) {
                    stopWhistleDetection();
                    stopSelf();
                }
            }
        } else {

            startWhistleDetection();
            Debug.e("", "intent is null OR intent.getExtras() is null");
        }
    } catch (Exception e) {
        e.printStackTrace();
    }

    return super.onStartCommand(intent, flags, startId);
}

private void startWhistleDetection() {

    try {
        stopWhistleDetection();
    } catch (Exception e) {
        e.printStackTrace();
    }

    recorderThread = new RecorderThread();
    recorderThread.startRecording();
    detectorThread = new DetectorThread(recorderThread);
    detectorThread.setOnWhistleListener(this);
    detectorThread.start();

}

private void stopWhistleDetection() {
    if (detectorThread != null) {
        detectorThread.stopDetection();
        detectorThread.setOnWhistleListener(null);
        detectorThread = null;
    }

    if (recorderThread != null) {
        recorderThread.stopRecording();
        recorderThread = null;
    }

}

@Override
public void onDestroy() {
    super.onDestroy();
}

@Override
public void onWhistle() {
    Debug.e("", "onWhistle()");
}

It detects whistle first time until you don't stop service. But after stopping and again starting It does not detect (does not call listener). I just failed to trace, what can be the issue?

Is there any issue with recording?

@ClassStacker I traced all variables. But if you want to trace variables of library of `music cg` then How it is possible?. — Bhavesh Hirpara, Feb 08 '13 at 07:43
Wouldn't the first thing which you have to check be the question why whistzle detection isn't _triggered_? Meaning, triggered from your code? — class stacker, Feb 08 '13 at 07:47
@ClassStacker `Debug.e("", "onWhistle()");` is called when initialize but I restart service it does not called. I tried with resetting everything (all objects). I think getting gramebytes does not returns `byte[]` properly. I tried with changing input channel also. — Bhavesh Hirpara, Feb 08 '13 at 07:56
I see... Can you provide a bit more information about how you stop and start? E.g. the code snippet from the Activity. Such that we have another perspective from which we can look at the code? — class stacker, Feb 08 '13 at 08:21
@BhaveshHirpara can you please send me the source code or any working example of musicg you have. the codes from github is not working for me. — Sagar Nayak, Jun 27 '16 at 05:00
Hello . i used above code and started service in mainactivity. But i always get averageAbsValue : 0.0 in public byte[] getFrameBytes() method of RecorderThread Class.......... and @Override public void onWhistle() { } i — KAMAL VERMA, Jun 28 '16 at 11:42

score 7 · Accepted Answer · answered Feb 08 '13 at 10:26

7

I invested 6 hours, :D Unbelievable, audio recorder is not released when it is stopped. I just released recorder after stopping.

Source code is having minor silly mistake. It is not releasing recorder.

public void stopRecording() {
    try {
        audioRecord.stop();
        audioRecord.release();

    } catch (Exception e) {
        e.printStackTrace();
    }
}

answered Feb 08 '13 at 10:26

Bhavesh Hirpara

22,255
15
63
104

this code is wonderful, could you please suggest me how to use this code, I am not able to understand thoroughly, went through service, I could understand but in case of recording and detection I can' get anything, it would be helpful if hint is given how to use this code. Thank you very much – Pending Intent Jun 30 '15 at 23:34
Hello Bhavesh Hirpara. i used above code and started service in mainactivity. But i always get averageAbsValue : 0.0 in public byte[] getFrameBytes() method of RecorderThread Class.......... and @Override public void onWhistle() { } is never triggered – KAMAL VERMA Jun 28 '16 at 11:41

score 0 · Answer 2 · answered Jun 04 '15 at 09:45

0

This code is ok for me

    if (detectorThread != null) {
            detectorThread.stopDetection();
            recorderThread.stopRecording();
    }

answered Jun 04 '15 at 09:45

Rosario Rapisarda

21
2

Detect 'Whistle' sound in android

2 Answers2

Linked