4

As a proof-of-concept I need to create a HTTP server which on GET request should start continuous stream of non-encoded/non-compressed audio data - WAV, PCM16. Let's assume the audio data are chunks of 4096 randomly generated mono audio samples @44.1kHz sampling rate.

What should I put in the HTTP response header in order to browser on the other end start a player in its UI for the user to listen in realtime?

I was reading about "Transfer-Encoding: chunked", "multipart", mimetype="audio/xwav", but still not sute what and how to use...

Great would be if someone can give me an exact example on Python/Flask because I'm not so confident with the web development.

PS1: Replacing HTTP server with an embedded device with limited HW power will be the next stage after the PoC.

PS2: This is the code which actually works and sends an WAV chunk as a single HTTP response:

from flask import Flask, Response,render_template
import pyaudio
import audio_processing as audioRec

app = Flask(__name__)

def genHeader(sampleRate, bitsPerSample, channels, samples):
    datasize = samples * channels * bitsPerSample // 8
    o = bytes("RIFF",'ascii')                                               # (4byte) Marks file as RIFF
    o += (datasize + 36).to_bytes(4,'little')                               # (4byte) File size in bytes excluding this and RIFF marker
    o += bytes("WAVE",'ascii')                                              # (4byte) File type
    o += bytes("fmt ",'ascii')                                              # (4byte) Format Chunk Marker
    o += (16).to_bytes(4,'little')                                          # (4byte) Length of above format data
    o += (1).to_bytes(2,'little')                                           # (2byte) Format type (1 - PCM)
    o += (channels).to_bytes(2,'little')                                    # (2byte)
    o += (sampleRate).to_bytes(4,'little')                                  # (4byte)
    o += (sampleRate * channels * bitsPerSample // 8).to_bytes(4,'little')  # (4byte)
    o += (channels * bitsPerSample // 8).to_bytes(2,'little')               # (2byte)
    o += (bitsPerSample).to_bytes(2,'little')                               # (2byte)
    o += bytes("data",'ascii')                                              # (4byte) Data Chunk Marker
    o += (datasize).to_bytes(4,'little')                                    # (4byte) Data size in bytes
    return o

FORMAT = pyaudio.paInt16
CHUNK = 102400 #1024
RATE = 44100
bitsPerSample = 16 #16
CHANNELS = 1
wav_header = genHeader(RATE, bitsPerSample, CHANNELS, CHUNK)

audio = pyaudio.PyAudio()

# start Recording
stream = audio.open(format=FORMAT, channels=CHANNELS,
    rate=RATE, input=True, input_device_index=10,
    frames_per_buffer=CHUNK)
# print "recording..."

@app.route('/')
def index():
    """Video streaming home page."""
    return render_template('index2.html')

@app.route('/audio_unlim')
def audio_unlim():
    # start Recording
    def sound():

        #while True:
        #    data = wav_header + stream.read(CHUNK)
        #    yield(data)
        data = wav_header + stream.read(CHUNK)
        yield(data)

    return Response(sound(),
                    mimetype="audio/x-wav")


if __name__ == "__main__":
    app.run(host='0.0.0.0', debug=True, threaded=True,port=5000)

and the index2.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Document</title>
</head>
<body>
    <audio controls>
        <source src="{{ url_for('audio_unlim') }}" type="audio/x-wav;codec=pcm">
        Your browser does not support the audio element.
    </audio
</body>
</html>

What to change in order to achieve continuous stream of chunks?

valioiv
  • 393
  • 2
  • 5
  • 15
  • Does this answer your question? [Audio Livestreaming with Python & Flask](https://stackoverflow.com/questions/51079338/audio-livestreaming-with-python-flask) – noslenkwah Nov 27 '19 at 17:31
  • @noslenkwah, no this does not solve my problem. The link you've refer to is about how to send one chunk of data as one shot with a single HTTP response. I want to to have continuous stream of audio data may be by using "Transfer-Encoding: chunked" or "Content-Transfer-Encoding: multipart". Still not sure the HTTP sequence diagram how to do that and what to type in the HTTP headers and MIME to notify browser to start corresponding player to interpret the RIFF data in the WAV header correctly... – valioiv Nov 27 '19 at 18:52

3 Answers3

3

Chunked transfer encoding is recommended, since the resource has an indefinite length. Without it, you would need to specify a Content-Length header. Older clients used to not be able to handled chunked transfer encoding well, so the old hack was to either leave out the Content-Length header entirely (HTTP/1.0 behavior), or to specify a very large (effectively infinite) length.

As for Content-Type, you can use audio/vnd.wav;codec=1 for regular PCM.

Be sure to set preload="none" on your <audio> element so that the browser doesn't try to buffer things ahead of time.

Brad
  • 159,648
  • 54
  • 349
  • 530
  • can you tell me what to change where in the code above? I've updated my issue with the exact code I'm using... – valioiv Nov 27 '19 at 19:05
  • @valioiv No, sorry, I'm no Python guy. Please do set the correct `Content-Type` though as I suggested. (Right now, you have `audio/x-wav`, which might work, but who knows.) I imagine Flask is going to handle chunked transfer encoding for you? – Brad Nov 27 '19 at 19:08
  • I assume, Flask handles chunked transfer by passing an iterator into Response – Arthur Grigoryan Nov 28 '19 at 09:29
3

Actually I've made a kind of workaround with the following code (without any index.html) and it works fine without any interruptions:

from flask import Flask, Response,render_template
import pyaudio
import audio_processing as audioRec

app = Flask(__name__)

def genHeader(sampleRate, bitsPerSample, channels, samples):
    datasize = 10240000 # Some veeery big number here instead of: #samples * channels * bitsPerSample // 8
    o = bytes("RIFF",'ascii')                                               # (4byte) Marks file as RIFF
    o += (datasize + 36).to_bytes(4,'little')                               # (4byte) File size in bytes excluding this and RIFF marker
    o += bytes("WAVE",'ascii')                                              # (4byte) File type
    o += bytes("fmt ",'ascii')                                              # (4byte) Format Chunk Marker
    o += (16).to_bytes(4,'little')                                          # (4byte) Length of above format data
    o += (1).to_bytes(2,'little')                                           # (2byte) Format type (1 - PCM)
    o += (channels).to_bytes(2,'little')                                    # (2byte)
    o += (sampleRate).to_bytes(4,'little')                                  # (4byte)
    o += (sampleRate * channels * bitsPerSample // 8).to_bytes(4,'little')  # (4byte)
    o += (channels * bitsPerSample // 8).to_bytes(2,'little')               # (2byte)
    o += (bitsPerSample).to_bytes(2,'little')                               # (2byte)
    o += bytes("data",'ascii')                                              # (4byte) Data Chunk Marker
    o += (datasize).to_bytes(4,'little')                                    # (4byte) Data size in bytes
    return o

FORMAT = pyaudio.paInt16
CHUNK = 1024 #1024
RATE = 44100
bitsPerSample = 16 #16
CHANNELS = 1
wav_header = genHeader(RATE, bitsPerSample, CHANNELS, CHUNK)

audio = pyaudio.PyAudio()

# start Recording
stream = audio.open(format=FORMAT, channels=CHANNELS,
    rate=RATE, input=True, input_device_index=10,
    frames_per_buffer=CHUNK)
# print "recording..."

@app.route('/audio_unlim')
def audio_unlim():
    # start Recording
    def sound():
        data = wav_header
        data += stream.read(CHUNK)
        yield(data)
        while True:
            data = stream.read(CHUNK)
            yield(data)

    return Response(sound(), mimetype="audio/x-wav")


if __name__ == "__main__":
    app.run(host='0.0.0.0', debug=True, threaded=True,port=5000)

I've just started with sending a WAV header, but the size written there is veeery big number telling the player to wait veeery big buffer of data. Until the "end" player plays the coming chunks of data without any problem (no WAV headers anymore just chunks of audio data!). This is without any "Transfer-encoding: chunked" or anything else! Just set the mimetype to "audio/x-wav". And the HTTP response is extremely simple as follows:

enter image description here

valioiv
  • 393
  • 2
  • 5
  • 15
-1

Server-side Streaming Technologies

In order to stream live audio, you will need to run specific streaming software on your server.



  • Icecast

    The Icecast server is an open source technology for streaming media. Maintained by the Xiph.org Foundation, it streams Ogg Vorbis/Theora as well as MP3 and AAC format via the SHOUTcast protocol.

    Note: SHOUTcast and Icecast are among the most established and popular technologies, but there are many more streaming media systems available.


Edit

I'm a Django guy and I've been testing things, and, seems, it works fine, only needs some proper file management and stuff. I've been working with mp3, but you can work with anything browsers have support for.

from django.http import StreamingHttpResponse

def stream(request):
    return StreamingHttpResponse(streamer(200000) ,content_type='audio/mp3')

def streamer(pointer):
    with open('media/Indila - Parle A Ta Tete.mp3', 'rb') as file:
        file.seek(pointer)
        for chunk in iter(lambda: file.read(4096), b''):
            yield chunk
#the connection is open until this iterator hasn't finished
Arthur Grigoryan
  • 358
  • 3
  • 12
  • It would be hard, even impossible, to run complex technologies like SHOUTcast and Icecast on the embedded device later on. I'd like to see minimalistic example with exact HTTP response headers and whatever metadata is needed to notify Chrome browser to start its audio player for WAV PCM16 and play it continuously. – valioiv Nov 27 '19 at 12:13
  • Then, you're gonna have to plunge into the low level browsers's [Web Audio API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API), or maybe make use of a pretty famous JS library like [howler.js](https://howlerjs.com/). – Arthur Grigoryan Nov 27 '19 at 13:03
  • 2
    @ArthurGrigoryan This answer is not correct at all. What valioiv is looking for is doable, and is in fact very similar to the way SHOUTcast/Icecast work. – Brad Nov 27 '19 at 18:41
  • Yeah, I'm sorry. I thought audio needs to be fragmented before streaming as it is with video, I suppose. – Arthur Grigoryan Nov 28 '19 at 09:24