Robust Download Function in PHP?

Question

I'm trying to download an image file programatically from within PHP and then treat it locally.

Edited: the previous function was replace by the one suggested below.

I have this function:

function downloadFile ($url, $path) {
    $result = false;
    $userAgent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)';

    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_USERAGENT, $userAgent); 
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 60); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); 
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

    curl_setopt($ch, CURLOPT_NOBODY, true);
    curl_setopt($ch, CURLOPT_HEADER, true);
    curl_exec( $ch ) ;
    if(!curl_errno($ch))  
    {
        $type = curl_getinfo($ch, CURLINFO_CONTENT_TYPE); 
        if ( stripos($type, 'image') !== FALSE )
        {
            // probably image

            curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
            curl_setopt($ch, CURLOPT_NOBODY, false);  
            curl_setopt($ch, CURLOPT_HEADER, false);    
            $fp=fopen($path,'wb');
            curl_setopt($ch, CURLOPT_FILE, $fp); 
            curl_exec($ch);
            fclose($fp);
            if ( exif_imagetype($path) != FALSE )
            {
                // 100% image
                $result = true;
            }
            else
            {
                // not an image
                unlink($path);
            }
        }
    }

    curl_close($ch);
    return $result;
}

What I really need is a function which is robust and can deal with any type of image and also if the url is invalid and there is no image.

update:

I changed my downloadFile function with the one suggested below. On my local computer it works great, but on my server it fails :/ I'm having some files downloaded with 0 bytes.

update2:

Still no progress, in the server for some reason the files are not downloaded. Besides having curl, is there any other requirements for it to ran in the server? I get also a "2006 - MySQL server has gone away", which I believe is caused by the download problem.

I was trying to do it to solve the problem of https urls not working, but it didn`t solve the problem. — Emerson, May 13 '13 at 05:54
this is the error: Warning [2] fopen() [function.fopen]: Unable to find the wrapper "https" - did you forget to enable it when you configured PHP? - Line: 971 - File: E:\wamp\www\mybb\inc\plugins\topposts.php PHP 5.3.13 (WINNT) — Emerson, May 13 '13 at 05:59
Doing a replace like that is a horrible idea — what if someone wanted to fetch, for example, `http://www.example.com/https-indicator.png`? — icktoofay, May 13 '13 at 06:02
Have you tried [libcurl](http://stackoverflow.com/questions/6476212/save-image-from-url-with-curl-php)? — nimrodm, May 13 '13 at 06:03
Yes the replacement was just for testing, and forgot it there. — Emerson, May 13 '13 at 06:10
Actually what I need is a function that can download from https and that could deal with invalid image urls — Emerson, May 13 '13 at 06:12
I configured PHP to use https, but I still get: Fatal error: Maximum execution time of 30 seconds exceeded in E:\wamp\www\mybb\inc\plugins\topposts.php on line 971 — Emerson, May 13 '13 at 06:16
The message "did you forget to enable it when you configured PHP?" means configuration at compile time. You can't configure it afterwards with php.ini. — Gerald Schneider, May 13 '13 at 06:26
use set_time_limit(0) function to set script execution time at the top of the script. — monish, May 13 '13 at 06:27
Yes, the one andrey gave below. Although I have the php_curl.dll and it is configured in php.ini (extension=php_curl.dll) it still doesnt work saying "Fatal error: Call to undefined function curl_init() in E:\wamp\www\mybb\inc\plugins\topposts.php on line 994" — Emerson, May 13 '13 at 06:46
To check that your PHP supports openssl, try running (from the command line) `php -i`. Search for "Registered Stream Socket Transports". It should say something like "tcp, udp, unix, udg, ssl, sslv3, sslv2, tls" — nimrodm, May 15 '13 at 06:52
Yes Andrew, I did Hi nimrodm, that's exactly what it says here: Registered Stream Socket Transports => tcp, udp, unix, udg, ssl, sslv3, sslv2, tls — Emerson, May 22 '13 at 05:19

score 1 · Answer 1 · answered May 13 '13 at 06:24

1

Use cURL.

This function checks an url for an image also. Returns true/false (image or not).

function downloadFile ($url, $path) {
    $result = false;
    $userAgent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)';

    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_USERAGENT, $userAgent); 
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 60); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); 
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

    curl_setopt($ch, CURLOPT_NOBODY, true);
    curl_setopt($ch, CURLOPT_HEADER, true);
    curl_exec( $ch ) ;
    if(!curl_errno($ch))  
    {
        $type = curl_getinfo($ch, CURLINFO_CONTENT_TYPE); 
        if ( stripos($type, 'image') !== FALSE )
        {
            // probably image

            curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
            curl_setopt($ch, CURLOPT_NOBODY, false);  
            curl_setopt($ch, CURLOPT_HEADER, false);    
            $fp=fopen($path,'wb');
            curl_setopt($ch, CURLOPT_FILE, $fp); 
            curl_exec($ch);
            fclose($fp);
            if ( exif_imagetype($path) != FALSE )
            {
                // 100% image
                $result = true;
            }
            else
            {
                // not an image
                unlink($path);
            }
        }
    }

    curl_close($ch);
    return $result;
}

Request:

if ( !downloadFile($url, $path) )
{
     // an error
}

answered May 13 '13 at 06:24

Andrey Volk

3,513
2
17
29

Thanks Andrey, but for some reason I can`t activate the curl extension on my wamp, and I can't put in production without testing locally :( – Emerson May 13 '13 at 06:43
What reason? Check libeay32.dll and ssleay32.dll exist. Enable php_openssl module also. – Andrey Volk May 13 '13 at 06:48
Yes they both exist. In both apache\bin and also in php5.3.13 folders. – Emerson May 13 '13 at 06:50
Try run empty php.exe from the command line. Does it show any message? What does php log say? – Andrey Volk May 13 '13 at 06:52
check your extension_dir in php.ini – Andrey Volk May 13 '13 at 06:55
My extension dir is "extension_dir = "e:/wamp/bin/php/php5.3.13/ext/" And I follow this instructions here: http://www.o3n.org/2012/06/wamp-server-2-2-windows-7-64-bit-and-curl-also-for-windows-8/ But to no avail :/ – Emerson May 13 '13 at 07:12
Thanks Andrey, that worked! I had managed to fix it yesterday with a similar approach. – Emerson May 17 '13 at 03:45
Hi Andrey, the function works fine on my local computer (under WAMP), but ont he server it gives permission error when trying to download the URLs. – Emerson May 19 '13 at 15:57
Check your premissions for the path where you save images. – Andrey Volk May 19 '13 at 16:01
I did, even with 777 it doesn't work. It fails on the line: $fp=fopen($path,'wb'); – Emerson May 19 '13 at 23:45
The permission problem was related to the backslaches, which is fixed now. But still I am having a problem when running the script on the server. Locally it downloads and resizes the images without a problem. – Emerson May 20 '13 at 04:05
Hi Andrey, still stuck, I believe the problem is downloading the images... Just not sure why the problem happens only at the server :/ – Emerson May 22 '13 at 06:06

score 0 · Accepted Answer · edited Jan 06 '14 at 11:31

At the end, this is the download function that worked best:

/*
 * This function downloads the image to the plugin/topposts/temp folder
 */
function downloadFile ($url, $path) {
 $ch = curl_init($url);
 curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
 $con = curl_exec($ch);
 curl_close($ch);
 file_put_contents($path, $con);
 return true;
}

Doesn't check for the type, but works. Thanks for the help.

score -1 · Answer 3 · answered May 13 '13 at 06:21

-1

I think you need to have openssl installed and configured with PHP.

answered May 13 '13 at 06:21

monish

182
2
10

Robust Download Function in PHP?

3 Answers3