Downloading big files and writing it locally

Question

which is the best way to download from php large files without consuming all server's memory?

I could do this (bad code):

$url='http://server/bigfile';
$cont = file_get_contents($url);
file_put_contents('./localfile',$cont);

This example loads entry remote file in $cont and this could exceed memory limit.

Is there a safe function (maybe built-in) to do this (maybe stream_*)?

Thanks

duplicate: http://stackoverflow.com/questions/4000483/php-how-download-big-file-using-php-low-memory-usage — KJYe.Name, Feb 15 '11 at 21:06
Yes they are. The accepted answer (poster used fsockopen) is one valid solution for that type of thing in PHP. Not because you don't think the answers aren't good that it justifies creating a duplicate of a question. You have enough rep, why not start a bounty? — netcoder, Feb 15 '11 at 21:41

score 6 · Accepted Answer · edited Dec 16 '17 at 16:30

6

You can use curl and the option CURLOPT_FILE to save the downloaded content directly to a file.

set_time_limit(0);
$fp = fopen ('file', 'w+b');
$ch = curl_init('http://remote_url/file');
curl_setopt($ch, CURLOPT_TIMEOUT, 75);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_exec($ch);
curl_close($ch);
fclose($fp);

edited Dec 16 '17 at 16:30

hanshenrik

19,904
4
43
89

answered Feb 15 '11 at 21:08

GWW

43,129
11
115
108

why do you set timeout and returntransfer? ([source](http://www.phpriot.com/articles/download-with-curl-and-php)) – dynamic Feb 15 '11 at 21:20
@yes123 actually you are right RETURNTRANSFER is probably not necessary. I put the timeout there for reference, it may be worth using it if you don't want your script to run forever. (Ie. maybe the network speed is slower than expected). There may be a default value for that is too low for your transfer time as well. So it may be worthwhile setting that to a high value or 0. – GWW Feb 15 '11 at 21:24

score 6 · Answer 2 · answered Feb 15 '11 at 21:25

6

Here is a function I use when downloading large files. It will avoid loading the entire file into the buffer. Instead, it will write to the destination as it receives the bytes.

function download($file_source, $file_target) 
{
    $rh = fopen($file_source, 'rb');
    $wh = fopen($file_target, 'wb');
    if (!$rh || !$wh) {
        return false;
    }

    while (!feof($rh)) {
        if (fwrite($wh, fread($rh, 1024)) === FALSE) {
            return false;
        }
    }

    fclose($rh);
    fclose($wh);

    return true;
}

answered Feb 15 '11 at 21:25

John Cartwright

5,109
22
25

I think this is a good function. At this point the choice is between a function like this or using curl direct stream write – dynamic Feb 15 '11 at 21:41
The difference is cURL will load the entire file into memory prior to writing (unless my memory has betrayed me). – John Cartwright Feb 15 '11 at 21:43
HM i don't think.. with `curl_setopt($ch, CURLOPT_FILE, $fp);` you can avoid memory usage – dynamic Feb 15 '11 at 21:53
I'm curious why you accepted the cURL solution when it does not address your primary concern, i.e., uses more memory. *Shrug* – John Cartwright Feb 15 '11 at 21:56
Read my comment above. If you set `CURLOPT_FILE` you are avoiding memory usage – dynamic Feb 16 '11 at 01:03

Downloading big files and writing it locally

2 Answers2

Related