PHP HTTP 1.1 can not perform partial PUT requests

Question

I have a file upload form on my website and when the user uploads its file to my server, I want to transport this uploaded file from my server to the cloud server. File can be large, so I decided to send the file to the cloud with 100KB portions.

Here is my PHP code sample which tries to send the file from my server to the cloud:

<?php
$cloud_server = '%my_server%';
$uploading_file_name = '%folders%/file.mp3';
$auth_token = '%token%';

$file_path = $_SERVER["DOCUMENT_ROOT"] . '/file.mp3';
$file_handler = fopen($file_path, 'rb');
$file_size = filesize($file_path);
$chunk_size = 100 * 1024;

$start_byte_position = 0;

while (!feof($file_handler)) {
    $chunk_content = fread($file_handler, $chunk_size);
    $chunk_content_length = strlen($chunk_content);
    $final_byte_position = $start_byte_position + $chunk_content_length;

    $socket_handler = fsockopen($cloud_server, 80, $errno, $errstr);
    if (!$socket_handler)
        die ($errstr);

    //Headers
    $request = "PUT /{$uploading_file_name} HTTP/1.1\r\n";
    $request .= "Connection: Close\r\n";
    $request .= "Host: {$cloud_server}\r\n";
    $request .= "X-Auth-Token: {$auth_token}\r\n";
    $request .= "Content-Range: {$start_byte_position}-{$final_byte_position}/{$file_size}\r\n";
    $request .= "Content-Length: {$chunk_content_length}\r\n";
    $request .= "\r\n";

    //Body
    $request .=  $chunk_content;
    $request .= "\r\n";

    fwrite($socket_handler, $request);

    $server_response = '';
    while (!feof($socket_handler))
        $server_response .= fgets($socket_handler, 128);
    echo $server_response, '<br><br>';

    $start_byte_position = $final_byte_position;
    fclose($socket_handler);
}

That code practically works, except one thing - each portion overwrites previous content. On the first loop's iteration 100KB file is created on the cloud, on the second iteration these 100KB are replaced with another 100KB. On the last iteration approximately 28KB are uploading, so finally I have 28KB file on the cloud's server instead of whole file. What am I doing wrong? I want these parts to follow each other, not to replace.

Here are server responses if that would help:

HTTP/1.1 201 Created Date: Tue, 13 Aug 2013 15:11:09 GMT Connection: close Server: Selectel_Storage/1.0 last-modified: Tue, 13 Aug 2013 15:11:09 GMT content-length: 0 etag: ea64231f21c952cdb57a5d3109415d09 content-type: text/html; charset=UTF-8

HTTP/1.1 201 Created Date: Tue, 13 Aug 2013 15:11:10 GMT Connection: close Server: Selectel_Storage/1.0 last-modified: Tue, 13 Aug 2013 15:11:09 GMT content-length: 0 etag: 01ab42f6ad58b401284540d6631dae9d content-type: text/html; charset=UTF-8

HTTP/1.1 201 Created Date: Tue, 13 Aug 2013 15:11:10 GMT Connection: close Server: Selectel_Storage/1.0 last-modified: Tue, 13 Aug 2013 15:11:10 GMT content-length: 0 etag: 99bd6f0d3cefd75abc140b9359464d6d content-type: text/html; charset=UTF-8

HTTP/1.1 201 Created Date: Tue, 13 Aug 2013 15:11:10 GMT Connection: close Server: Selectel_Storage/1.0 last-modified: Tue, 13 Aug 2013 15:11:10 GMT content-length: 0 etag: 07cf64ef10e96710b8cb48022ee5dd16 content-type: text/html; charset=UTF-8

................................

HTTP/1.1 201 Created Date: Tue, 13 Aug 2013 15:11:33 GMT Connection: close Server: Selectel_Storage/1.0 last-modified: Tue, 13 Aug 2013 15:11:33 GMT content-length: 0 etag: f01e66c96b21665bbbd19ee5e283a4e1 content-type: text/html; charset=UTF-8

Thanks in advance!

Not possible. See http://stackoverflow.com/questions/2364110/whats-the-justification-behind-disallowing-partial-put — Marc B, Aug 13 '13 at 15:26
OK, I'll watch, but whan can you say about the second answer from here? http://stackoverflow.com/questions/716680/difference-between-content-range-and-range-headers If I understand correctly, this is exactly what I need. — Kirill Smirnov, Aug 13 '13 at 15:28
you can REQUEST a range of bytes from the server. content-range is what the server actually gives you in response to that request (it can decide to send less than requested). Just because those particular headers exist doesn't mean you slap them into a PUT request and start doing partial puts. For exactly the reasons in the answer I linked to. The headers are for requesting data FROM the server, not for sending data TO the server. — Marc B, Aug 13 '13 at 15:39
Hmm, but is there a way to somehow perform partial file uploading? I understand now that it's impossible through PUT request, but it's hard to believe that this is impossible at all. — Kirill Smirnov, Aug 13 '13 at 15:45

score 1 · Accepted Answer · edited May 23 '17 at 10:31

Well, I finally found an appropriate solution. As Marc B responded, it's impossible to perform partial PUT request (because this type of request must be idempotent by definition). However, if we speak about uploading files to the huge cloud server, it is possible to use appropriate server's API. As far as I can judge, OpenStack (its Swift component to be precise) is popular enough among file storages.

So here is brief and clear official documentation.

Cloud server, which I use, allows uploading large files only with DLO (Dynamic Large Objects) method, so all you need is:

1) Split your large file into segments (or simply use fseek and fread).

2) Send these segments to one specified directory, which name will further be considered as whole file's name.

3) Send special request (create so-called manifest file), imploding all downloaded segments in one file.

Quite easy!

Here is a good example from the documentation page using CURL :

# First, upload the segments
curl -X PUT -H 'X-Auth-Token: <token>' \
    http://<storage_url>/container/myobject/1 --data-binary '1'
curl -X PUT -H 'X-Auth-Token: <token>' \
    http://<storage_url>/container/myobject/2 --data-binary '2'
curl -X PUT -H 'X-Auth-Token: <token>' \
    http://<storage_url>/container/myobject/3 --data-binary '3'

# Next, create the manifest file
curl -X PUT -H 'X-Auth-Token: <token>' \
    -H 'X-Object-Manifest: container/myobject/' \
    http://<storage_url>/container/myobject --data-binary ''

# And now we can download the segments as a single object
curl -H 'X-Auth-Token: <token>' \
    http://<storage_url>/container/myobject

Hope this would help somebody.

PHP HTTP 1.1 can not perform partial PUT requests

1 Answers1