I am trying to develop a music recognition web application using ACRCloud audio recognition.
I have a client side in React that records the music from the laptop's microphone and sends it to the server side:
const sendRecording = async (dispatch, arrayBuffer) => {
const config = {
headers: {
'Content-Type': 'multipart/form-data; boundary=${data._boundary}'
}
};
try {
var formData: FormData = new FormData();
formData.append('audio', arrayBuffer);
const res = await axios.post(
'/api/recording',
{ test: new Buffer(arrayBuffer) },
config
);
} catch (err) {
console.log(err);
}
};
const handleRecorder = dispatch => {
navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
console.log('entra!!!!');
const mediaRecorder = new MediaRecorder(stream);
mediaRecorder.start();
const audioChunks: any[] = [];
mediaRecorder.addEventListener('dataavailable', event => {
audioChunks.push(event.data);
});
mediaRecorder.addEventListener('stop', async () => {
const audioBlob = new Blob(audioChunks);
let arrayBuffer = await new Response(audioBlob).arrayBuffer();
sendRecording(dispatch, arrayBuffer);
});
setTimeout(() => {
mediaRecorder.stop();
}, 13000);
});
};
This apparently works for me and sends correctly the buffer. I get something like this on the backend:
<Buffer 7b 22 61 75 64 69 6f 22 3a 7b 22 74 79 70 65 22 3a 22 42 75 66 66 65 72 22 2c 22 64 61 74 61 22 3a 5b 32 36 2c 36 39 2c 32 32 33 2c 31 36 33 2c 31 35 ... >
The Node backend looks like this:
import express = require('express');
import { identifyAudio } from '../../services/acousticFingerprintService';
import fs = require('fs');
const router = express.Router();
import multer from 'multer';
let processMultipart = multer({ storage: multer.memoryStorage() });
const WavEncoder = require('wav-encoder');
router.post('/', processMultipart.array('audio'), (req, res) => {
let buffer: Buffer = new Buffer(0);
req.on('data', data => {
buffer = Buffer.concat([buffer, data]);
});
req.on('end', () => {
const sound = {
sampleRate: 44100,
channelData: [
new Float32Array(buffer)
]
};
WavEncoder.encode(sound).then(bufferWav => {
identifyAudio(bufferWav);
});
res.send('Recording sent');
});
});
The audio is sent to the ACRCloud recognition service throught the export const identifyAudio = bitmap => { ... } function
The flow works but I always get a 'No result' response. I have tried to send an audio stored on the server. It is a recorded audio with low quality and noise to simulate the real scenario.
let filename = '../../services/sample.wav';
let bitmap: Buffer = fs.readFileSync(path.resolve(__dirname, filename));
Sending that bitmap I receive a correct response with the name of the artist, song ...
I used to do identifyAudio(buffer) and I always got a 'Can't generate fingerprint' response.
I encoded it with wav-encoder and I get the 'No result' response. So I guess I am sending a well constructed audio.
