Audio Livestreaming with Python & Flask

Question

I'm currently strugling with my implementation of a simple live streaming web application using Python and Flask. It seems that I'm not able to stream my live recorded audio from the servers microphone input to the webpage.

server.py

from flask import Flask, render_template, Response
import cv2
import framework
import pyaudio
import audio_processing as audioRec

FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100
CHUNK = 1024

audio = pyaudio.PyAudio()


app = Flask(__name__)


@app.route('/')
def index():
    """Video streaming home page."""
    return render_template('index.html')


# Stream routing
@app.route('/video_feed')
def video_feed():
    """Video streaming route. Put this in the src attribute of an img tag."""
    return Response(generateVideo(),
                    mimetype='multipart/x-mixed-replace; boundary=frame')


@app.route("/audio_feed")
def audio_feed():
    """Audio streaming route. Put this in the src attribute of an audio tag."""
    return Response(generateAudio(),
                    mimetype="audio/x-wav")


# Stream generating
def generateVideo():
    """Video streaming generator function."""
    cap = cv2.VideoCapture(0)
    while (cap.isOpened()):
        ret, frame = cap.read()
        output = framework.streamer(frame, 'final')
        cv2.imwrite('signals/currFrame.jpg', output)
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + open('signals/currFrame.jpg', 'rb').read() + b'\r\n')


def generateAudio():
    """Audio streaming generator function."""
    currChunk = audioRec.record()
    data_to_stream = genHeader(44100, 32, 1, 200000) + currChunk
    yield data_to_stream

    # with open("signals/audio.wav", "rb") as fwav:
    #     data = fwav.read(1024)
    #     while data:
    #         yield data
    #         data = fwav.read(1024)


def genHeader(sampleRate, bitsPerSample, channels, samples):
    datasize = samples * channels * bitsPerSample // 8
    o = bytes("RIFF",'ascii')                                               # (4byte) Marks file as RIFF
    o += (datasize + 36).to_bytes(4,'little')                               # (4byte) File size in bytes excluding this and RIFF marker
    o += bytes("WAVE",'ascii')                                              # (4byte) File type
    o += bytes("fmt ",'ascii')                                              # (4byte) Format Chunk Marker
    o += (16).to_bytes(4,'little')                                          # (4byte) Length of above format data
    o += (1).to_bytes(2,'little')                                           # (2byte) Format type (1 - PCM)
    o += (channels).to_bytes(2,'little')                                    # (2byte)
    o += (sampleRate).to_bytes(4,'little')                                  # (4byte)
    o += (sampleRate * channels * bitsPerSample // 8).to_bytes(4,'little')  # (4byte)
    o += (channels * bitsPerSample // 8).to_bytes(2,'little')               # (2byte)
    o += (bitsPerSample).to_bytes(2,'little')                               # (2byte)
    o += bytes("data",'ascii')                                              # (4byte) Data Chunk Marker
    o += (datasize).to_bytes(4,'little')                                    # (4byte) Data size in bytes
    return o




if __name__ == '__main__':
    app.run(host='0.0.0.0', debug=True, threaded=True)

audio_processing.py

import pyaudio

FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100
CHUNK = 1024

audio = pyaudio.PyAudio()


def record():
    # start Recording
    stream = audio.open(format=FORMAT, channels=CHANNELS,
                    rate=RATE, input=True,
                    frames_per_buffer=CHUNK)
    # print "recording..."
    data = stream.read(CHUNK)
    return data

I'm trying to get the current Chunk of the microphone using the audio_processing.py and using yield to respond the current samples to the user. The video stream works quite good. Anyone knows, what I'm doing wrong here?

Kind regards, Felix

Just a quick note, you shouldn't be continuously opening the PyAudio stream; i.e. set it up in an init function call or something similar, before the streaming starts. Then access the stream object to read from it when you need with your `record()` method. Or at least close it if you want to keep opening it. — WoodyDev, Jun 28 '18 at 09:50
Yes, your are right, I changed the code a bit. I recently added the pyAudio recording procedure to the server.py into generateAudio(). The function should work properly right now. The only problem is, that the byte() function is not supported in Python2.7 which I am currently using. Do you have any idea, how to write a similar function in 2.7? Couldn't find a premade solution :/ — F. Geißler, Jun 28 '18 at 14:26
Hello @F.Geißler , I wanted to see how did you call the @app.route("/audio_feed") endpoint in your html . New to this setup . Will be of great help . — Rajesh Rajamani, Apr 21 '21 at 13:10

score 14 · Answer 1 · edited Mar 24 '22 at 00:40

Here is a working example with the inbuilt microphone of your device. Sorry for not being able to explain much, but here's what I figured out for my app!

app.py

from flask import Flask, Response,render_template
import pyaudio

app = Flask(__name__)


FORMAT = pyaudio.paInt16
CHANNELS = 2
RATE = 44100
CHUNK = 1024
RECORD_SECONDS = 5

 
audio1 = pyaudio.PyAudio()
 


def genHeader(sampleRate, bitsPerSample, channels):
    datasize = 2000*10**6
    o = bytes("RIFF",'ascii')                                               # (4byte) Marks file as RIFF
    o += (datasize + 36).to_bytes(4,'little')                               # (4byte) File size in bytes excluding this and RIFF marker
    o += bytes("WAVE",'ascii')                                              # (4byte) File type
    o += bytes("fmt ",'ascii')                                              # (4byte) Format Chunk Marker
    o += (16).to_bytes(4,'little')                                          # (4byte) Length of above format data
    o += (1).to_bytes(2,'little')                                           # (2byte) Format type (1 - PCM)
    o += (channels).to_bytes(2,'little')                                    # (2byte)
    o += (sampleRate).to_bytes(4,'little')                                  # (4byte)
    o += (sampleRate * channels * bitsPerSample // 8).to_bytes(4,'little')  # (4byte)
    o += (channels * bitsPerSample // 8).to_bytes(2,'little')               # (2byte)
    o += (bitsPerSample).to_bytes(2,'little')                               # (2byte)
    o += bytes("data",'ascii')                                              # (4byte) Data Chunk Marker
    o += (datasize).to_bytes(4,'little')                                    # (4byte) Data size in bytes
    return o

@app.route('/audio')
def audio():
    # start Recording
    def sound():

        CHUNK = 1024
        sampleRate = 44100
        bitsPerSample = 16
        channels = 2
        wav_header = genHeader(sampleRate, bitsPerSample, channels)

        stream = audio1.open(format=FORMAT, channels=CHANNELS,
                        rate=RATE, input=True,input_device_index=1,
                        frames_per_buffer=CHUNK)
        print("recording...")
        #frames = []
        first_run = True
        while True:
           if first_run:
               data = wav_header + stream.read(CHUNK)
               first_run = False
           else:
               data = stream.read(CHUNK)
           yield(data)

    return Response(sound())

@app.route('/')
def index():
    """Video streaming home page."""
    return render_template('index.html')

      
if __name__ == "__main__":
    app.run(host='0.0.0.0', debug=True, threaded=True,port=5000)

index.html - under templates folder in current directory

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Document</title>
</head>
<body>
    <audio controls>
        <source src="{{ url_for('audio') }}" type="audio/x-wav;codec=pcm">
        Your browser does not support the audio element.
    </audio>
</body>
</html>

Try not to post code only answer but also explain the OP what was wrong with his code — dWinder, May 08 '19 at 09:44
Hi, I am trying to set this up and I have been able to get it to play, but there is a loud square-wave sound overlaying the audio. Pyaudio recording to file with the same encoding seems to not have the effect, so I think there is something wrong with the streaming post-recording. Any input on what this might be would be helpful. I have tried changing the number of channels and tried all of the input devices on my machine. — Mr. Negi, Jan 15 '20 at 21:24
You are only supposed to send the header once; this code is sending the header with every chunk. This caused the "loud square-wave sound" that @Mr.Negi noticed. — zzxx53, Feb 03 '20 at 18:57
I applied the change you suggested and verified that it worked, I have submitted an edit request with the change to the answer, as I don't have sufficient reputation to edit without confirmation. Thanks for providing your insight here, as a non-specialist in audio streaming, I had no idea that this was the problem. — Mr. Negi, Feb 05 '20 at 19:23

Audio Livestreaming with Python & Flask

1 Answers1

Linked