0

I'm working on a simple app that scans all URLs on a page and display http status code of every URls. When the URLs is more than 50 I got an error Fatal error: Maximum execution time of 30 seconds exceeded , what I did is, add this line of code ini_set('max_execution_time', 300);. It work, my problem is I have to wait until it finish then display the whole result, is there a way to make it optimize or display the result while it is scanning.

Thanks.

CODE

$html = file_get_contents('http://www.example.com');

    $dom = new DOMDocument();
    @$dom->loadHTML($html);

    // grab all the on the page
    $xpath = new DOMXPath($dom);
    $hrefs = $xpath->evaluate("/html/body//a");

    for ($i = 0; $i < $hrefs->length; $i++) {
           $href = $hrefs->item($i);
           $url = $href->getAttribute('href');
           echo $url.'-'. statusCode($url) .'<br />';
    }

function statusCode($url) { 

    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_NOBODY, true);
    curl_exec($ch);
    $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);

    return $httpCode;

}
user123
  • 435
  • 1
  • 6
  • 23
  • 1
    ***display the result...*** Where are you displaying the results? php shell? If so, you should see the results on the go... – Romeo Sierra Dec 20 '18 at 03:24
  • You need asynchronous CURL implementation here...check this for reference.. https://stackoverflow.com/questions/36171222/async-curl-request-in-php – Mangesh Sathe Dec 20 '18 at 03:41
  • @RomeoSierra , just on the page, It takes time to see the result while checking the http codes. – user123 Dec 20 '18 at 03:47
  • In that case there is nothing much that you could do. Execution of php happens in the php engine and after everything is done only, the result is returned to the web server. So the server will wait for the results. – Romeo Sierra Dec 20 '18 at 03:50
  • @user123, if some answer was helpful, mark it as right. – BSeitkazin Dec 20 '18 at 05:18

2 Answers2

1

PHP makes http request synchronously. To achieve your expected result you should make them asynchronously. I suggest you to use Guzzle library. The simple async request looks like:

$request = new \GuzzleHttp\Psr7\Request('GET', 'http://example.com');
$promise = $client->sendAsync($request)->then(function ($response) {
    echo 'I completed! ' . $response->getBody();
});

So, making more than 50 http requests asynchronously, you just get result, when it finishes.

BSeitkazin
  • 2,889
  • 25
  • 40
0

Yes because php need to end the process to display them. Only after all process done php display results in the page. As @Romeo Sierra said, if it's in php shell, it will display as it go with process. I think you want to display it in a page right? You can do this,

  • Get data to create the for loop in to the page and create a Javascript loop from it.
  • In that loop create a ajax call to get one data result according to the index of the loop.
  • Ajax success append data to a list or table.
  • Results will display as process go with it.

Hope this will work.

damnguy
  • 95
  • 1
  • 11
  • you can't just call js code inside loop, to do it async, you need node.js instance at least. – BSeitkazin Dec 20 '18 at 04:50
  • @BSeitkazin You didn't get what I said.I said get data to page to create js loop and in that loop create ajax. What you need is only get `$hrefs->length` to the page to create js loop. – damnguy Dec 20 '18 at 05:18
  • 1
    JavaScript runs in the UI thread of the browser and any long running process will lock the UI, making it unresponsive. Additionally, there is an upper limit on the execution time for JavaScript and the browser will ask the user whether to continue the execution or not. All of this is really bad user experience. The user won't be able to tell whether everything is working fine or not. Furthermore, the effect will be worse for users with a slow connection. – BSeitkazin Dec 20 '18 at 05:32
  • Yes @BSeitkazin that's 100% correct. I just help with a simple way to do it. Because **user123** mentioned _"I'm working on a simple app that scans all URLs on a page and display http status code of every URls"_ It's better your way for long run. Node is better than Guzzle right? But little bit worka nd study to do if new user. – damnguy Dec 20 '18 at 05:47