Download large file to server (Remote Upload) in chunk with PHP and AJAX (with time limit forced setting)

Question

There's a question here (on StackOverflow) that asks about streaming large files to user in chunks. A referred to code, originally here, in an answer to this question tells how. I'm looking for how to just save the file to server.

Notes:

• The script here aims to download a file to server by providing URL to download file from (This process is also named remote upload).
• My server provider disabled me from editing time limit, so downloads using this script takes time.
• I am able to save file contents to server using file_put_contents("MyFile.iso",$buffer,FILE_APPEND), but not the whole file, mostly because the script takes long time running so it times out.
• I think a solution may work like so: a JavaScript method requests PHP actions in the background via AJAX multiple times, the first background request tells PHP to download the first 100MB of the file. The second request tells PHP to download the second 100MB of the file, and so on till the PHP tells Javascript that we reached to the end of the file. So instead we downloaded the file in one whole process (long time taking), we downloaded it on multiple processes (small time taking).
  • A good start: How to partially download a remote file with cURL? (I will find time later soon to develop the whole solution altogether. Any help will be appreciated.

Below is the mentioned code that I need to start with in order to save/remote-upload file to server: (edited: it now saves the file to server, but not the whole file, mostly because the script takes long time running)

<?php
define('CHUNK_SIZE', 1024*1024); // Size (in bytes) of tiles chunk

// Read a file and display its content chunk by chunk
function readfile_chunked($fileurl, $retbytes = TRUE) {
    $buffer = '';
    $cnt    = 0;
    $handle = fopen($fileurl, 'rb');

    if ($handle === false) {
        return false;
    }

    while (!feof($handle)) {
        $buffer = fread($handle, CHUNK_SIZE);
        file_put_contents("MyFile.iso",$buffer,FILE_APPEND);
        ob_flush();
        flush();

        if ($retbytes) {
            $cnt += strlen($buffer);
        }
    }

    $status = fclose($handle);

    if ($retbytes && $status) {
        return $cnt; // return num. bytes delivered like readfile() does.
    }

    return $status;
}

$fileurl = 'http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-desktop-amd64.iso';
$mimetype = 'mime/type';
header('Content-Type: '.$mimetype );
readfile_chunked($fileurl);

?>

Answer 1

I knew this question is old, but I hope my answer help others.

The question is not actually 'Download Large File To Server', but it should be Upload Large File To Server instead.

To upload a large file to a server, there are many ways. Here's is how I do it using FileReader and XMLHttpRequest.

https://stackoverflow.com/a/49808460/6348813

The idea is, you need to read the file as a Binary (ReadAsBinaryString()) or ArrayBuffer (ReadAsArraBuffer()) then you can stream the file to a server. In PHP, to listen the streamed Binary or ArrayBuffer is simply using php://input as the listen directory.

You need to consider doing this under HTTPS or your connection will be open to attacker.

The other method, you may try upload the file using the slice() method. This method has some good performance as it support the pause, close and resume upload whenever the connection is not stable.

In my experience, the ReadAsArrayBuffer() method seems more faster than ReadAsBinaryString() method and slice() method even the connection is under 100kbps.

All above methods has some common features, it is, 'You don't need to setting your PHP upload limit'.

Note:

DO NOT USE ReadAsBinaryString() METHOD AS IT HAS BEEN DEPRECATED IN MOZILLA AND DO NOT SUPPORT LARGE FILE MORE THAN 300MB.

This feature is non-standard and is not on a standards track. Do not use it on production sites facing the Web: it will not work for every user. There may also be large incompatibilities between implementations and the behavior may change in the future.

Original Article

https://developer.mozilla.org/en-US/docs/Web/API/FileReader

https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest