Http range header requests entire file

Question

I am working with the c# webserver from codeplex version 1.1. I have implemented the Accept-Range headers and it does work. However when I use wireshark (Version 1.4.1 (SVN Rev 34476 from /trunk-1.4)) to catch the traffic, I see the following:

GET /movies/i_am_legend%20dvd/main.m4v HTTP/1.1
Host: 10.100.1.199:8081
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
Connection: keep-alive
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
X-Playback-Session-Id: 9CED81CC-BFAE-4CF6-A477-0EA62B2C652F

HTTP/1.1 206 PartialContent
Content-Range: bytes 0-1/652965648
Accept-Ranges: bytes
ETag: "0daA8D4/wgt4MFvxdNIPLw=="
Date: Wed, 13 Jun 2012 09:10:18 GMT
Content-Length: 2
Content-Type: video/x-m4v
Server: Tiny WebServer
Connection: keep-alive

..  << 2 bytes data

GET /movies/i_am_legend%20dvd/main.m4v HTTP/1.1
Host: 10.100.1.199:8081
Accept: */*
Range: bytes=0-652965647
Accept-Encoding: identity
Connection: keep-alive
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
X-Playback-Session-Id: 9CED81CC-BFAE-4CF6-A477-0EA62B2C652F

HTTP/1.1 206 PartialContent
Content-Range: bytes 0-652965647/652965648
Accept-Ranges: bytes
ETag: "0daA8D4/wgt4MFvxdNIPLw=="
Date: Wed, 13 Jun 2012 09:10:18 GMT
Content-Length: 652965648
Content-Type: video/x-m4v
Server: Tiny WebServer
Connection: keep-alive

The webserver will try to send the entire file ( >600MB), wireshark shows that the entire conversation is 159774 bytes. If I do the same thing with IIS I get similar headers

GET /ipod/main.m4v HTTP/1.1
Host: 10.100.1.199
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
X-Playback-Session-Id: C5BBF91D-78AB-42BA-ACE0-D74AB9D845CE
Connection: keep-alive

HTTP/1.1 206 Partial Content
Content-Type: video/x-m4v
Last-Modified: Mon, 11 Jun 2012 10:33:41 GMT
Accept-Ranges: bytes
ETag: "7243cabbd47cd1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Wed, 13 Jun 2012 09:21:03 GMT
Content-Length: 2
Content-Range: bytes 0-1/652965648

..  << 2 bytes of data

GET /ipod/main.m4v HTTP/1.1
Host: 10.100.1.199
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
Accept: */*
Range: bytes=0-652965647
Accept-Encoding: identity
X-Playback-Session-Id: C5BBF91D-78AB-42BA-ACE0-D74AB9D845CE
Connection: keep-alive

HTTP/1.1 206 Partial Content
Content-Type: video/x-m4v
Last-Modified: Mon, 11 Jun 2012 10:33:41 GMT
Accept-Ranges: bytes
ETag: "7243cabbd47cd1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Wed, 13 Jun 2012 09:21:03 GMT
Content-Length: 652965648
Content-Range: bytes 0-652965647/652965648

Wireshark shows that the entire conversation is 175615 bytes.

I have searched for more information on the Accept-Range headers, and so far I can only find that the server must send the requested range. But I can't believe that it was meant to use a range request for requesting a huge file in one time.

My webserver tries to send the entire file because it has been requested as such, but I see new range requests coming in with more huge ranges like this (only the Range header copied from the request header. The (@time ... ) is the time of wireshark

Range: bytes=2162688-652965647 (@ time == 1.646204)
Range: bytes=4980736-652965647 (@ time == 2.754322)
Range: bytes=6356992-652965647 (@ time == 2.922479)

After reading this I have tried to send a shorter range whenever I get the range request for the entire file. But then it does not work at all.

I would like to know:

Is the range request for the entire file is some kind of bug in iOS (seen it with 4.3.3 as well) I would have expected Range: bytes=0-1 and after the replay something like Range: bytes=0-65535/652965648
Can I somehow gracefully deny this large request and tell the requested that I can deliver a maximum size at once? (I did not find this in the RFC)
Is IIS simply aborting this request after certain amount of bytes?

EDIT: For number 3: Not IIS but the browser seems to simply aborting (and closing) the connection. After that making a new request. I can't imagine that the Range Request was meant to request the entire file or HUGE parts of the file.

EDIT: In iOS7 it seems to have changed. The first range request is still the same (bytes 0-1). After that, I see 2 or 3 range requests as mentioned above, where the last request keeps on transferring bytes for a longer period. However still multiple requests are done.

I am [observing similar behaviour](http://stackoverflow.com/questions/12637728/http-byte-range-protocol-client-behaviour-on-ipad-iphone), and to me it looks like this is iPad/iPhone issue. — mindas, Oct 03 '12 at 09:10
@mindas Indeed you;re facing the same thing. Due to the structure of the webserver that I use, I not have much choice and need to re-throw the exception to stop handling the range request. Fortunately I only have 1 client connection at the same time (but it also works with 2 or 3). I hope you will get some useful answers. — bart s, Oct 03 '12 at 12:34
what we see in our testing is safari asks for the first two bytes (range 0-1), then it asks for the entire file and immediately closes the connection, then it asks for the *last few hundred kilobytes of the file* (which I suspect contains important metadata. Chrome does the same thing by the way), and then after that it starts asking for the file from start to finish in small chunks. I suspect the exact pattern will vary depending on the video format. — Abhi Beckert, Jul 13 '15 at 06:31
@AbhiBeckert : thanks for your feedback. I have never seen the range request asking for bytes at the end of the file, but indeed as you mention, it may be depending on the video file format. We are using .m4v files. Can you feedback what type of files you use? — bart s, Jul 13 '15 at 07:18

score 5 · Accepted Answer · edited Jan 30 '18 at 13:05

Is the range request for the entire file is some kind of bug in iOS (seen it with 4.3.3 as well) I would have expected Range: bytes=0-1 and after the replay something like Range: bytes=0-65535/652965648

I don't know whether it is a bug. However, I can think of reasons for a media player to request the entire file in one request. In this way the media player gets a data stream from which it can read all data from start to finish.

As soon as the media player have read enough data from the stream it can start playing the media file. It then chooses how much more data it shall buffer in the background while the media is playing. There could be several different approaches to this:

Eagerly buffer the entire media file. This is a good strategy when bandwidth is cheap (user is not paying or paying a flat rate for data transfer). It is assumed that the user will want to see/listen to the entire media file.
Lazily buffer just enough to avoid lagging. This is a good strategy when bandwidth is expensive (user is paying by the byte).

In an ideal setup, the media player wouldn't have to buffer anything at all and instead decode data from the stream while playing the media in real time. However, that would require that the underlying network channel would be super stable and transfer data at the required pace at all times.

This is not the case, and therefore the media player will choose to buffer a couple of seconds or minutes ahead.

It is important to note that whatever strategy is chosen it could still make sense for the media player to request the entire resource in a single request.

However, range requests are vital to the media player when:

The connection is aborted (for any reason).
The user jumps ahead in the media. (for example, wants to see what's 10 minutes into a movie)

The media player can then close the data stream it originally opened and send a range request for the desired position.

Can I somehow gracefully deny this large request and tell the requested that I can deliver a maximum size at once?

No you cannot. Range requests are initiated by the client/browser and a server that have stated that it support range requests (via Accept-Ranges header) must obey the client and respond with whatever range it requests.

What you can do however is to send data with a Transfer-Encoding: chunked header. This will enable your server to control how large chunks of data it shall transmit. However, it is still done over a single HTTP connection.

Thanks for extensive reaction. I will mark it as answer. – bart s Sep 29 '14 at 09:36 — bart s, Sep 29 '14 at 09:36

Http range header requests entire file

1 Answers1