Android, live amplitude and pitch detection

Question

I'm writing an app that needs to listen to the microphone and give me a live amplitude and pitch output. I have figured out how to do pitch recognition. I've been doing a lot of research into fft. found the Android library TarsosDSP which makes listening for pitch extremely simple:

AudioDispatcher dispatcher = 
        AudioDispatcherFactory.fromDefaultMicrophone(22050,1024,0);
PitchDetectionHandler pdh = new PitchDetectionHandler() {
    @Override
    public void handlePitch(PitchDetectionResult res, AudioEvent e){
        final float pitchInHz = res.getPitch();
        runOnUiThread(new Runnable() {
            @Override
            public void run() {
                processPitch(pitchInHz);
            }
        });
    }
};
AudioProcessor pitchProcessor = new PitchProcessor(PitchEstimationAlgorithm.FFT_YIN, 22050, 1024, pdh);
dispatcher.addAudioProcessor(pitchProcessor);

Thread audioThread = new Thread(dispatcher, "Audio Thread");
audioThread.start();

I've also figured out how to do amplitude detection by just using the build-in android .getMaxAmplitude() method.

but my problem is that I can't for the life of me figure out how to do both at the same time. the problem is that you apparently can run more than one instance of the microphone. As in if you try to run two separate live recordings on separate threads. I've looked trough the entire internet trying to look for some example code to get me going , but I can't find a thing. Has anyone ever had to do anything similar?

edit I've figured out that you can use the AudioEvent from the Pitchdetectionhandler. audioevent.getbytebuffer() returns a byte array with the audio data in bytes, according to the documentation: https://0110.be/releases/TarsosDSP/TarsosDSP-latest/TarsosDSP-latest-Documentation/ .

and if I'm not mistaking when converted to a short[], the highest value is the highest amplitude right?

but:

final byte[] audioBytes = e.getByteBuffer();
 short[] shortArray = new short[audioBytes.length];
         for (int index = 0; index < audioBytes.length; index++) {
                    shortArray[index] = (short) audioBytes[index];
                            float item = shortArray[index];
                               if (item > amp){
                                        amp = item;
                                    }
                                }

in this case amp always returns 127. And this method wouldn't really work live anyway?

so three more questions. is my basic thinking right, if so why does it always return 127, and how would I use this in a live context?

1. List item

Answer 1

found the solution myself. you can do audioEvent.getFloatBuffer() then run that buffer trough some fft methods and then you can extract the amplitude value from the buffer. the buffer is pretty small so I just ended up getting the max amplitude from that buffer while its running, that'll get you an amplitude readout many times a second and that's live enough for me. edit example:

 public void handlePitch(PitchDetectionResult result, final AudioEvent e) {

                        final float pitchInHz = result.getPitch();
                        final float[] amplitudes = new float[e.getBufferSize()];

                        new Thread(new Runnable() {
                            public void run() {
                                if (pitchInHz > pitch) {
                                    pitch = pitchInHz;
                                }

                                float[] audioFloatBuffer = e.getFloatBuffer();
                                float[] transformBuffer = new float[e.getBufferSize() * 2];
                                FFT fft = new FFT(e.getBufferSize());
                                System.arraycopy(audioFloatBuffer, 0, transformBuffer, 0, audioFloatBuffer.length);
                                fft.forwardTransform(transformBuffer);
                                fft.modulus(transformBuffer, amplitudes);


                                for (int index = 0; index < amplitudes.length; index++) {
                                    if (amplitudes[index] > amp) {
                                        amp = amplitudes[index];
                                    }
                                }
                            }
                        }).start();
                    }