Browsers close socket before the response is fully downloaded

Question

I have a simple Python server that uses http.server. The goal is not to show the video in a html page, or to download the video file, but to display the video in the browser directly. This is what I have so far:

import http.server

class SuperHandler(http.server.SimpleHTTPRequestHandler):
    def do_GET(self):
        path = self.path
        encodedFilePath = 'file.mp4'

        with open(encodedFilePath, 'rb') as videoFile:
            self.send_response(200)
            self.send_header('Content-type', 'video/mp4')
            self.end_headers()
            self.wfile.write(videoFile.read())
            print('File sent: ' + videoFile.name)

server_address = ('', 8000)
handler_class = SuperHandler
httpd = http.server.HTTPServer(server_address, handler_class)
httpd.serve_forever()

The problem that I have is that the response doesn't contain the full video. file.mp4 is 50MB, but when I look in the network tab of either Chrome or Firefox, it says that the response is only 1MB. Is there a reason why the full file isn't transferred? Do I need to add some sort of HTTP header to make this work?

EDIT:

This is my code now:

server_address = ('', 8000)
handler_class = http.server.SimpleHTTPRequestHandler
httpd = http.server.HTTPServer(server_address, handler_class)

httpd.serve_forever()

I am now using the default SimpleHTTPRequestHandler's do_GET, but it's still not working (although the response is now 40MB/30MB instead of 1MB).

When I request file.mp4 on Chrome, the socket connexion is closed after ~7 seconds (~5 seconds on Firefox), which makes the script throw a BrokenPipeError: [Errno 32] Broken pipe, because the server is still trying to write the rest of the video file on a closed socket.

So my question is: how can I make the browser download the full response before it closes the socket?

Additional info

HTTP Response headers sent to the client:

HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.5.0
Date: Mon, 28 Dec 2015 02:36:39 GMT
Content-type: video/mp4
Content-Length: 53038876
Last-Modified: Fri, 25 Dec 2015 02:09:52 GMT

Why do you want to use `http.server` to serve static files? You could run: `python3 -mhttp.server` to serve files in a directory (assuming you use it as a quick-n-dirty hack). — jfs, Dec 25 '15 at 02:44
Serving static files is just a part of what I want to do, so I need to code it myself. — Maxime Dupré, Dec 25 '15 at 02:47
If you are determined to use `http.server` here, look at at how `http.server.SimpleHTTPRequestHandler` is implemented (`copyfile()` reads the file from disk and sends it to the user). — jfs, Dec 25 '15 at 02:53
Where exactly can I find the implementation code of SimpleHTTPRequestHandler? — Maxime Dupré, Dec 25 '15 at 15:16
Found it! http://www.opensource.apple.com/source/python/python-3/python/Lib/SimpleHTTPServer.py — Maxime Dupré, Dec 25 '15 at 15:18
The official mercurial repository is at [http://hg.python.org/](https://hg.python.org/cpython/file/tip/Lib/http/server.py) — jfs, Dec 25 '15 at 15:31
I get the same problem using just the SimpleHTTPRequestHandler. It seems like the browser closes the TCP connexion before the file has been fully downloaded. The python code triggers "BrokenPipeError: [Errno 32] Broken pipe". However, when I use only the SimpleHTTPRequestHandler (no inheritance), the browser can download about 30MB of the 50MBs, but that's still not good enough. Maybe there is something special to do when sending large files e.g. a http header I have to add. — Maxime Dupré, Dec 25 '15 at 17:57
`SimpleHTTPRequestHandler` works (I've just tested it on a 1GB file using `curl url | wc -c` command). Do you need help with creating [an html page that uses the video file](https://en.wikipedia.org/wiki/HTML5_video#.3Cvideo.3E_element_examples)? — jfs, Dec 25 '15 at 18:05
It woks fine when the request type is "document" - when the file is being directly downloaded on your disk by the browser. However, when the request type is "media" (e.g. "video/mp4"), which is used to display a video directly in the browser, just as you can view a pdf in a browser, the browser seems to close the connexion, thus the video file isn't completely downloaded. I am not trying to show the video file in an html file, I am trying to show the video directly in the browser, using it's built-in player. — Maxime Dupré, Dec 25 '15 at 18:44
It seems like after ~7 seconds, Chrome closes the connexion and Firefox after ~5 seconds. I tested multiple times, I'm gonna look into this... — Maxime Dupré, Dec 25 '15 at 18:49
you should change your question: "how to **stream** a video file so that it can be played using Chrome, Firefox" — jfs, Dec 25 '15 at 19:07
But is it streaming that I am attempting to do? Am I not trying to simply serve a video file? Maybe it's the same thing. — Maxime Dupré, Dec 26 '15 at 23:12
I don't know what you are trying to do. Whatever you are trying to do your code is broken for large files (don't use `.read()` that loads the whole file into memory), use `copyfile` as I've mentioned above (to read files in chunks). — jfs, Dec 27 '15 at 02:58
I am trying to make the browser display a video. I am not using the code above anymore, I am just using the default SimpleHTTPRequestHandler. It still causes a `BrokenPipeError: [Errno 32] Broken pipe`, when accessing to a large .mp4. — Maxime Dupré, Dec 27 '15 at 15:50
Add a [Content-Length](http://stackoverflow.com/q/2773396/208880) header to let the receiver know how much data to expect. — Ethan Furman, Dec 28 '15 at 02:16
Doesn't work, I already have that header sent to the client. — Maxime Dupré, Dec 28 '15 at 02:17
@maximedupre I wonder is it because the mp3 file is too large? — BAE, Dec 28 '15 at 04:58
if you need Range support then have a look at https://github.com/danvk/RangeHTTPServer — ccpizza, Mar 12 '23 at 15:19

score 3 · Accepted Answer · answered Jan 02 '16 at 02:54

It turns out that the video file I was using was corrupted (maybe it had some dropped frames or something like that). I tested with multiple other .mp4 and it worked like a charm.

All you really need to play a video file (or more accurately stream a video file as pointed by @hrunting) as far as http headers for the response are concerned is:

HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.5.0
Date: Sat, 02 Jan 2016 02:45:34 GMT
Content-type: video/mp4
Content-Length: 33455269
Last-Modified: Sat, 02 Jan 2016 02:45:27 GMT

I think even the Server, Date and Last-Modified headers aren't mandatory (they are sent automatically by SimpleHTTPRequestHandler).

As pointed by @qarma and @hrunting, if you want to enable the user to jump to a specific time in video, you should support the Range header. It's simply a good idea to support the Range header as it is sent by default by Chrome.

score 1 · Answer 2 · answered Dec 28 '15 at 10:51

1

To stream videos you should support at least range requests and transfer-encoding: chunked

As far as I can see http.server does not support that directly. You can of course implement that on top.

Alternatively use a simple framework, e.g. bottle (one file, already supports both) or cherrypy (more solid, multi-threaded, etc)

You can also get by without any Python code, e.g. if you use nginx

answered Dec 28 '15 at 10:51

Dima Tisnek

11,241
4
68
120

But is it streaming that I am attempting to do? Am I not trying to simply serve a video file? Maybe it's the same thing. – Maxime Dupré Dec 28 '15 at 19:14
Also why do the browsers download for only 5/7 seconds? – Maxime Dupré Dec 28 '15 at 19:34
1

I don't see why I would need to support range requests, as I want to transfer the entire file and I also don't see why I would need to support transfer-encoding: chunked, as the server already provides a Content-Length. – Maxime Dupré Dec 28 '15 at 20:41
@maximedupre capture network traffic against a reference server and go from there. Range makes sense if user may jump to a specific time in video. Broken pipe suggests that browsers wants less data than is sent, quite possible if you ignored range header. – Dima Tisnek Dec 29 '15 at 08:22
http://ronallo.com/blog/html5-video-everything-i-needed-to-know/ and also "buffering" section of http://www.chromium.org/audio-video – Dima Tisnek Dec 29 '15 at 08:31

score 1 · Answer 3 · answered Jan 01 '16 at 20:44

Fundamentally, @qarma is right. To stream videos, you need to use a library or framework that supports, at the very least, Range: headers.

But, you are not /trying/ to stream video. The browser is doing it for you. When you return the video/mp4 content type, the browser, which knows how to stream video itself, immediately switches to a streaming mode. It stops the file download it had going (the source of your pipe errors), and restarts with a Range: header. It leverages existing media player code within the browser to do so. Since the SimpleHTTPServer class doesn't support the Range: header, it doesn't handle the response properly.

If you want to prevent this streaming behavior and force the browser to download the file without playing it, then return the Content-Disposition header forcing it to treat the video file as a downloaded file rather than as a piece of content to be rendered.

Here's some code based on your initial question that does what you want:

import http.server

class SuperHandler(http.server.SimpleHTTPRequestHandler):
    def do_GET(self):
        path = self.path
        encodedFilePath = 'file.mp4'

        with open(encodedFilePath, 'rb') as videoFile:
            self.send_response(200)
            self.send_header('Content-type', 'video/mp4')
            self.send_header('Content-Disposition', 'attachment; filename=' + encodedFilePath)
            self.end_headers()
            self.copyfile(videoFile, self.wfile)

server_address = ('', 8000)
handler_class = SuperHandler
httpd = http.server.HTTPServer(server_address, handler_class)
httpd.serve_forever()

Theoretically, you also send back Accept-Ranges: none, but Chrome, at least, seems to ignore that.

I still don't see why I would need to support the `Range` headers. I always want the full video file to be transferred, not just part of the video. Maybe you are saying that because as @qarma said, "the user may jump to a specific time in video", but I don't think it has anything to do with my problem (although thanks for the tip). When you say: "It stops the file download it had going (the source of your pipe errors), and restarts with a Range: header", you are right, but it only occurs in Chrome (not Firefox). And no, I don't want to download the video file, I want to play it in the browser. — Maxime Dupré, Jan 02 '16 at 02:42
I have actually found the solution to my problem, I will post it as an answer, but thank you for the help. — Maxime Dupré, Jan 02 '16 at 02:43

Browsers close socket before the response is fully downloaded

3 Answers3