Using Curl and PHP

Question

I use PHP to grab a list of urls from the database. Each url is checked by Check_URL Function in php that uses curl to check if the website is live.

function Check_URL($url) {

    $agent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
    $ch = curl_init();
    curl_setopt ($ch, CURLOPT_URL, $url);
    curl_setopt ($ch, CURLOPT_USERAGENT, $agent);
    curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt ($ch, CURLOPT_TIMEOUT, 3);
    $page = curl_exec($ch);
    $httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

    curl_close($ch);
    if ($httpcode >= 200 && $httpcode <= 301) {
         return true;
    } else { 
        return false;
    }
}

Unfortunately, the page takes forever to load. I have read curl_multi_init(); might be my solution but I'm not sure how I can implement it into what I have. Thanks in advance.

So when you load a page on your server, this function is called and checks some number of urls? that means that not only is your page loading, but you're also waiting for X urls to be loaded by CURL. How many urls are you loading? You may want to look into having something else trigger your script, like an image on your page. — Aerik, Mar 13 '12 at 23:02
This is the sort of thing that you should really do in the background, separate from the thread that's rendering the page for the user. Throw tasks for these URLs into beanstalkd, and write a worker to test them and store the results in your db. — Frank Farmer, Mar 13 '12 at 23:04

score 0 · Answer 1 · answered Mar 13 '12 at 23:02

0

After you initialize the curl_multi_init, you need to iterate through your URLs in a for or foreach loop. YOu'll find a good example here: http://www.fusionswift.com/examples/2011/08/php-curl_multi_exec-example/

For your function, I'd have it handle an array of URLs as the parameter.

answered Mar 13 '12 at 23:02

SBerg413

14,515
6
62
88

1

Sounds like good input, save the overhead of re-initializing, but he's still going to be loading 30 urls... I think he needs to have an image on his page load the php script to check the urls or use ajax or something. – Aerik Mar 13 '12 at 23:07
1

@Aerik - You're absolutely right - if the page is waiting on all of these URLs to load or be tested, that's not a good UI experience for the end user. An alternative could be to load the initial page, and then handle this functionality in an AJAX call, while the user is given some type of informative message. – SBerg413 Mar 13 '12 at 23:12

score 0 · Answer 2 · edited May 23 '17 at 12:31

0

Use also CURLOPT_CONNECTTIMEOUT option.

Your function can work faster if you set CURLOPT_NOBODY to TRUE.

Look this code if you want to check multiple urls.

edited May 23 '17 at 12:31

Community

1
1

answered Mar 26 '12 at 21:43

Mikhail Gerasimov

36,989
16
116
159

Using Curl and PHP

2 Answers2