Standard method for HTTP partial upload, resume upload

Question

I'm developing http client/server framework, and looking for the correct way to handle partial uploads (the same as for downloads using GET method with Range header).

But, HTTP PUT is not intended to be resumed. And PATCH method, as i know, doesn't accept Range header.

Is there any way to handle this in by HTTP standard (not using extension headers or etc)?

Answer 1

I think there is no standard for partial uploads:

• Content-Range inside requests is not explicitly forbidden in RFC2616 (http), but also the wording refers to it as an response header which gets used in response of a range-request
• while you could use the PATCH method to update an existing resource (e.g. to add more bytes) it would not be the same as a partial upload, because all the time the incomplete resource would be available

If you look at the protocols of Dropbox, google drive etc they all roll their own protocol to transfer a single files in multiple chunks. What you need for resumeable uploads is

• a way to address an incomplete upload. Normal URLs address a complete, not a partial resource and I know of no standard for partial resources.
• a way to find out the current state of the upload, maybe also checksums of the part to be sure, that the local file did not change. This could be provided by WebDAV PROPFIND method (once you are able to address the incomplete resource :)
• a way to upload a chunk. Here one could maybe use PATCH with a content-range header. mod_dav seems to allow PUT with content-range header (see http://www.gossamer-threads.com/lists/apache/users/432346)
• a way to publish the resource once it is complete, or a way to define upfront what complete means (e.g size of resource, checksum...)

Answer 2

As noted in some of the comments, newer versions of the HTTP specification have clarified this somewhat. Per Section 4.3.4 of RFC 7231:

An origin server that allows PUT on a given target resource MUST send a 400 (Bad Request) response to a PUT request that contains a Content-Range header field (Section 4.2 of [RFC7233]), since the payload is likely to be partial content that has been mistakenly PUT as a full representation. Partial content updates are possible by targeting a separately identified resource with state that overlaps a portion of the larger resource, or by using a different method that has been specifically defined for partial updates (for example, the PATCH method defined in [RFC5789]).

Unfortunately, the discussion of the range headers which occurs in RFC 7233 focuses more or less entirely on GET requests, and RFC 5789 defines almost nothing about PATCH except that it is specifically not required to transmit the entire content (but is allowed to), nor is it required to be idempotent (but is allowed to be).

The bright side is that because PATCH is so loosely defined, it does accommodate the approach given in an answer to a related question (https://stackoverflow.com/a/6711496/7467189): just change "PUT" to "PATCH". While there is no requirement that a server interpret a PATCH request with a Content-Range header this way, it is certainly a valid interpretation, just not one that can be relied upon from arbitrary servers or clients. But in cases such as the original question, where control of both ends is available, it is at least an obvious approach and does not violate the current standards.

One additional consideration is that the Content-Type should express whatever is being transmitted, rather than the content type of the entity as a whole (the RFC gives some examples in this regard). For content that is being "patched" in arbitrary chunks this would imply application/octet-stream, although there are scenarios where the client and server might be more content-aware and opt to send patches as entities which have a more specific definition (e.g. single pages of a multi-page image format).

Answer 3

PATCH would be a logical method to choose for resumable uploads: it expects a media type that indicates how to change the target resource. Though not specifically defined as format to perform patching, multipart/byteranges specifies a byte range and the contents of that range, making it suitably well defined for PATCH payloads.

Example:

PATCH /document HTTP/1.1
Content-Type: multipart/byteranges; boundary=THIS_STRING_SEPARATES

--THIS_STRING_SEPARATES
Content-Type: text/plain
Content-Range: bytes 10-21/22

1234567890
--THIS_STRING_SEPARATES--

This example uploads twelve bytes at a ten-byte offset. THIS_STRING_SEPARATES is an arbitrary, user-picked delimiter, and should be randomly generated. Some headers omitted for brevity, each line is terminated with ␍␊.

Answer 4

Use the Range xxxx-yyyy header or the Range xxxx- header with PUT to update part of a file. It is supported by Apache.

Don't be confused by the statement in RFC 7231 that Content-Range cannot be used. That is intended to prevent badness by clients taking headers received from the server and using PUT to send them back to the server. This caution notice is not relevant to the question of partials PUTs.