POST HTML5 audio data to server

Question

I am currently implementing a web application and I want the users to record some audio and then I want a submit button to POST the mp3 file recorded to the server.

My server (Flask)'s main route '/' is waiting for the POST request:

@app.route('/', methods=['GET', 'POST'])
def index():
  if request.method == "GET":
    return render_template('index.html', request="GET")
  else:
    print request.files
    print request.form
    print request.form['file']
    if 'file' not in request.files:
      flash('No file part')
      return redirect(request.url)
    file = request.files['file']
    if file.filename == '':
      flash('No selected file')
      return redirect(request.url)
    if file and allowed_file(file.filename):
      handle_file(file)
    return render_template('index.html', request="POST")

Here is my JS code:

Jsfiddle

There are two main issues here:

1) When I download the mp3 file after recording, it cannot be opened by a media player. It seems like I am doing something wrong in just recording the audio.

2) When I print request.form in my server after getting the POST request, I only get this:

ImmutableMultiDict([('file', u'')])

And print request.form['file'] returns an empty line.

Why is this happening? Is there something wrong with the POST request.

Finally, I want to be able to decode the string I am posting to convert back to mp3. How do I do that?

Note: None of this has to stay the same. The task is to record audio and then POST it to the server. If there is a more efficient way to do that, any tips are welcome. Also, I don't care if the file will be wav or mp3.

Answer 1

Note: this answer only treats current implementations in both chrome and Firefox. All this is subject to change any time soon.

---

I am not sure if anything is wrong in your server-side code, but don't send binary data as string. Instead, use an FormData to send it as multipart (you'll win 30% of data + integrity).

Also, it seems that in your MediaRecorder code, you are finalizing the file at every dataavailable event. It's generally not what you want.

---

Currently, no browser does support recording as mp3 natively.

var mimes = ['mpeg', 'mpeg3', 'x-mpeg3', 'mp3', 'x-mpeg']
console.log(mimes.some(m=>MediaRecorder.isTypeSupported('audio/'+m)));

So if you want to go the MediaRecorder way, you'd have to accommodate yourself with opus codec, encapsulated either in webm for chrome, or ogg for FF :

var enc = ['ogg', 'webm'];
var mime = "";
enc.forEach(e => {
  if (!mime && MediaRecorder.isTypeSupported(`audio/${e};codecs="opus"`)) {
    mime = `audio/${e};codecs="opus"`;
  }
});
console.log(mime);

So now we've got the correct mimeType, we can simply tell the browser to use it :

fiddle for chrome

var enc = ['ogg', 'webm'];
var extension = "",
  mime = '';
enc.forEach(e => !extension &&
  (mime = `audio/${e};codecs="opus"`) &&
  MediaRecorder.isTypeSupported(mime) &&
  (extension = e));
navigator.mediaDevices.getUserMedia({
    audio: true
  })
  .then(stream => {
    const chunks = [];
    const rec = new MediaRecorder(stream, {
      mimeType: mime // use the mimeType we've found
    });
    // this is not where we build the file, but where we store the chunks
    rec.ondataavailable = e => chunks.push(e.data);
    rec.onstop = e => {
      // stop our gUM stream
      stream.getTracks().forEach(t => t.stop());
      // NOW create the file
      let blob = new Blob(chunks, {
        type: mime
      });
      // we could now send this blob : 
      //   let form = new FormData();
      //   form.append('file', blob, 'filename.'+extension;
      //   ... declare xhr
      //   xhr.send(form);
      // but we'll just fetch it for the demo :
      let url = URL.createObjectURL(blob);
      let au = new Audio(url);
      au.controls = true;
      document.body.appendChild(au);
      au.play();
      // and create an downloadable link from it : 
      let a = document.createElement('a');
      a.href = url;
      a.download = 'filename.' + extension;
      a.innerHTML = 'download';
      document.body.appendChild(a);
    };
    rec.start();
    setTimeout(() => rec.stop(), 3000);
  });

Or we could also just let the browser do everything by default. This would be a problem only for the file extension...

---

Now, if you prefer wav over opus, you could let the MediaRecorder away, and simply use the WebAudioAPI, pass your gUM stream to it, and record the data from there. Here I'll use the recorder.js library for simplicity.

fiddle for chrome

navigator.mediaDevices.getUserMedia({
  audio: true
})
.then(stream => {
  const aCtx = new AudioContext();
  const streamSource = aCtx.createMediaStreamSource(stream);
  var rec = new Recorder(streamSource);
  rec.record();
  setTimeout(() => {
    stream.getTracks().forEach(t => t.stop());
    rec.stop()
    rec.exportWAV((blob) => {
      // now we could send this blob with an FormData too
      const url = URL.createObjectURL(blob);
      let au = new Audio(url);
      au.controls = true;
      document.body.appendChild(au);
      au.play();
      let a = document.createElement('a');
      a.href = url;
      a.innerHTML = 'download';
      a.download = 'filename.wav';
      document.body.appendChild(a);
    });
  }, 3000);
})

<script src="https://rawgit.com/mattdiamond/Recorderjs/master/dist/recorder.js"></script>

And if you really want an mp3, I guess you could use one of the javascript lame libraries available on the web.

Answer 2

Try Converting the audio blob to Base64 and post the base64 string to the server.

function submit(blob) {
  var reader = new window.FileReader();
  reader.readAsDataURL(blob);
  reader.onloadend = function() {

    var fd = new FormData();
    base64data = reader.result;
    fd.append('file', base64data, 'audio.mp3');

    $.ajax({
      type: 'POST',
      url: '/',
      data: fd,
      cache: false,
      processData: false,
      contentType: false,
      enctype: 'multipart/form-data'
    }).done(function(data) {
      console.log(data);
    });
  }

}

Now, Convert the base64 String to a binary stream in your server.

Form more information on how to decode Base64 in python checkout this post. Python base64 data decode