Capturing Sound for Analysis and Visualizing Frequencies in Android

Question

I'm new in Android and I'm trying to make a program which captures an audio sound and then displays the frequencies that exist within it. I found an example that draws the graphic portion of a graphic equalizer. In this example it is used an object of type AudioRecord to capture audio sound. The technique used to break an audio signal down into component frequencies employs a mathematic transformation called a discrete Fourier transform (DFT) and to perform DFT it is used a fast Fourier transform (FFT). This example use a package which implements the FFT. The package is linked here www.netlib.org/fftpack/jfftpack.tgz. The problem is that after I run this example the graphic equalizer doesn't appear on the display after I press the start button.

Here is the source code for the activity class:

package com.audio.processing;
import android.app.Activity;
import android.graphics.Bitmap;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.MediaRecorder;
import android.os.AsyncTask;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.ImageView;

import ca.uol.aig.fftpack.RealDoubleFFT;

public class AudioProcessing extends Activity implements OnClickListener{
    int frequency = 8000;
    int channelConfiguration = AudioFormat.CHANNEL_CONFIGURATION_MONO;
    int audioEncoding = AudioFormat.ENCODING_PCM_16BIT;


    private RealDoubleFFT transformer;
    int blockSize = 256;
    Button startStopButton;
    boolean started = false;

    RecordAudio recordTask;

    ImageView imageView;
    Bitmap bitmap;
    Canvas canvas;
    Paint paint;
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
        startStopButton = (Button) this.findViewById(R.id.StartStopButton);
        startStopButton.setOnClickListener(this);

        transformer = new RealDoubleFFT(blockSize);

        imageView = (ImageView) this.findViewById(R.id.ImageView01);
        bitmap = Bitmap.createBitmap((int)256,(int)100,Bitmap.Config.ARGB_8888);
        canvas = new Canvas(bitmap);
        paint = new Paint();
        paint.setColor(Color.GREEN);
        imageView.setImageBitmap(bitmap);
    }

    private class RecordAudio extends AsyncTask<Void, double[], Void> {
        @Override
        protected Void doInBackground(Void... params) {
        try {
            int bufferSize = AudioRecord.getMinBufferSize(frequency,
                    channelConfiguration, audioEncoding);
                    AudioRecord audioRecord = new AudioRecord(
                    MediaRecorder.AudioSource.DEFAULT, frequency,
                    channelConfiguration, audioEncoding, bufferSize);

                    short[] buffer = new short[blockSize];
                    double[] toTransform = new double[blockSize];
                    audioRecord.startRecording();
                    while (started) {
                    int bufferReadResult = audioRecord.read(buffer, 0, blockSize);

                    for (int i = 0; i < blockSize && i < bufferReadResult; i++) {
                        toTransform[i] = (double) buffer[i] / 32768.0; // signed 16 bit
                        }

                    transformer.ft(toTransform);
                    publishProgress(toTransform);
                    }
                    audioRecord.stop();
                    } catch (Throwable t) {
                    Log.e("AudioRecord", "Recording Failed");
                    }
                    return null;
                    }
        }

    protected void onProgressUpdate(double[]... toTransform) {
        canvas.drawColor(Color.BLACK);
        for (int i = 0; i < toTransform[0].length; i++) {
        int x = i;
        int downy = (int) (100 - (toTransform[0][i] * 10));
        int upy = 100;
        canvas.drawLine(x, downy, x, upy, paint);
        }
        imageView.invalidate();
        }

        public void onClick(View v) {
        if (started) {
        started = false;
        startStopButton.setText("Start");
        recordTask.cancel(true);
        } else {
        started = true;
        startStopButton.setText("Stop");
        recordTask = new RecordAudio();
        recordTask.execute();
        }
        }
}

Here is the main.xml :

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
>
<TextView
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:text="@string/hello"
/>
<ImageView android:id="@+id/ImageView01" android:layout_width="wrap_content"
android:layout_height="wrap_content"></ImageView><Button android:text="Start"
android:id="@+id/StartStopButton" android:layout_width="wrap_content"
android:layout_height="wrap_content"></Button>
</LinearLayout>

In the AndroidManifest.xml I set the RECORD_AUDIO permission. Thanks in advance !

where isthe image view tag in the xml ? – codeScriber Apr 01 '11 at 09:50 — codeScriber, Apr 01 '11 at 09:50

score 13 · Answer 1 · answered Aug 07 '12 at 07:48

Here is the working code. I tried it myself. It works fine.

package com.example.frequencytest;

import android.app.Activity;
import android.graphics.Bitmap;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.MediaRecorder;
import android.os.AsyncTask;
import android.os.Bundle;
import android.util.Log;
import android.view.Menu;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.ImageView;
import ca.uol.aig.fftpack.RealDoubleFFT;

public class MainActivity extends Activity implements OnClickListener {

    int frequency = 8000;
    int channelConfiguration = AudioFormat.CHANNEL_CONFIGURATION_MONO;
    int audioEncoding = AudioFormat.ENCODING_PCM_16BIT;
    private RealDoubleFFT transformer;
    int blockSize = 256;

    Button startStopButton;
    boolean started = false;

    RecordAudio recordTask;

    ImageView imageView;
    Bitmap bitmap;
    Canvas canvas;
    Paint paint;

    //AudioRecord audioRecord;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        startStopButton = (Button) this.findViewById(R.id.start_stop_btn);
        startStopButton.setOnClickListener(this);

        transformer = new RealDoubleFFT(blockSize);

        imageView = (ImageView) this.findViewById(R.id.imageView1);
        bitmap = Bitmap.createBitmap((int) 256, (int) 100,
                Bitmap.Config.ARGB_8888);
        canvas = new Canvas(bitmap);
        paint = new Paint();
        paint.setColor(Color.GREEN);
        imageView.setImageBitmap(bitmap);

    }

    public class RecordAudio extends AsyncTask<Void, double[], Void> {

        @Override
        protected Void doInBackground(Void... arg0) {

            try {
                // int bufferSize = AudioRecord.getMinBufferSize(frequency,
                // AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
                int bufferSize = AudioRecord.getMinBufferSize(frequency, 
                        channelConfiguration, audioEncoding); 

                AudioRecord audioRecord = new AudioRecord( 
                        MediaRecorder.AudioSource.MIC, frequency, 
                        channelConfiguration, audioEncoding, bufferSize); 

                short[] buffer = new short[blockSize];
                double[] toTransform = new double[blockSize];

                audioRecord.startRecording();

                // started = true; hopes this should true before calling
                // following while loop

                while (started) {
                    int bufferReadResult = audioRecord.read(buffer, 0,
                            blockSize);

                    for (int i = 0; i < blockSize && i < bufferReadResult; i++) {
                        toTransform[i] = (double) buffer[i] / 32768.0; // signed
                                                                        // 16
                    }                                       // bit
                        transformer.ft(toTransform);
                        publishProgress(toTransform);



                }

                audioRecord.stop();

            } catch (Throwable t) {
                t.printStackTrace();
                Log.e("AudioRecord", "Recording Failed");
            }
            return null;
        }

        @Override
        protected void onProgressUpdate(double[]... toTransform) {

            canvas.drawColor(Color.BLACK);

            for (int i = 0; i < toTransform[0].length; i++) {
                int x = i;
                int downy = (int) (100 - (toTransform[0][i] * 10));
                int upy = 100;

                canvas.drawLine(x, downy, x, upy, paint);
            }

            imageView.invalidate();

            // TODO Auto-generated method stub
            // super.onProgressUpdate(values);
        }

    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        getMenuInflater().inflate(R.menu.activity_main, menu);
        return true;
    }

    public void onClick(View arg0) {
        // TODO Auto-generated method stub
        if (started) {
            started = false;
            startStopButton.setText("Start");
            recordTask.cancel(true);
        } else {
            started = true;
            startStopButton.setText("Stop");
            recordTask = new RecordAudio();
            recordTask.execute();
        }
    }
}

After some experimenting... in main activity (just to make it easier to follow what happens, and to use HEIGHT where ever 100 was used): static final int HEIGHT = 100; static final int MIDDLE = HEIGHT/2; static final int PEAK = MIDDLE/10; //was HEIGHT/10 then in onProgressUpdate: int x = 0; for (int i = 0; i < toTransform[0].length; i++) { int v = (int) (MIDDLE + (toTransform[0][i]*PEAK)); final int top = Math.min(v, MIDDLE); final int bottom = Math.max(v, MIDDLE); canvas.drawLine(x, top, x, bottom, paint); x++; } and see what happens... — Larphoid, Mar 01 '13 at 22:52

score 3 · Answer 2 · answered Nov 24 '11 at 09:43

3

Yes, I also had this project and i had the same error as you but after adding the permission below all is okay now. Most probably you didn't add it in the correct place in the androidmanifest.xml.It should be outside the application tag.

  <uses-permission android:name="android.permission.RECORD_AUDIO">
     </uses-permission>

answered Nov 24 '11 at 09:43

Hussein

187
1
3
14

Also on Android 6 (Marshmallow) and later you need to ask user for permission. See https://stackoverflow.com/questions/49653692/startrecording-called-on-an-uninitialized-audiorecord – Anthony. Dec 10 '20 at 11:35

score 2 · Answer 3 · answered Jan 17 '12 at 16:59

2

onProgressUpdate method should belong to RecordAudio where as in your code it is belonging to AudioProcessing. Check the braces, it should work based on above correction

answered Jan 17 '12 at 16:59

Raghu

21
2

Capturing Sound for Analysis and Visualizing Frequencies in Android

3 Answers3

Linked