1

I'm trying to retrieve the HTML for a handful of websites. I was using PHP file_get_contents which worked really quite well but was slow and was a little limited so I decided to try Guzzle. Unfortunately I'm finding it really tough to stop Guzzle throwing exceptions even using their documented exception handling. My code is really simple:

    $websiteFQDN = 'http://acme.biz';

    $client = new \GuzzleHttp\Client(['http_errors' => false,'verify' => false, 'connect_timeout' => 121 ]);

    echo '<h3>'.$websiteFQDN.'</h3>';
    try {
        $res = $client->request('GET', $websiteFQDN);
        echo '<p>Appears Successful</p>';
    }
    catch (RequestException $e){
        echo '<p>Exception Caught</p>';
    }

I keep getting the following error:

GuzzleHttp\Exception\ConnectException cURL error 28: Failed to connect to acme.biz port 80: Timed out (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

Any suggestions on why the above isn't catching the exception and allowing the process to continue? In my testing it loops through about 20 domains with no problems but then I get the error above and kills it.

Laravel 7.0 | Guzzle 7.0.1 | PHP 7.2.5

Someone has recommended trying the Laravel HTTP Client but unfortunately it suffers from the same problems because it uses Guzzle which in turn appears to use CUrl, all I'm doing is adding layers of abstraction. I tried it with this code:

    $response = Http::withOptions(['verify' => false])->get($websiteFQDN);
    if($response->successful()){
        echo '<p>Appears Successful</p>';
    }else{
        echo '<p>Exception Caught</p>';
    }

When I try using Laravel HTTP vanilla, I get the "cURL error 60: SSL certificate" error mentioned here. I add the 'verify' => false Guzzle option to the HTTP Client which passes it to Guzze (just as a I did when I used Guzzle), that solves that and it gets through more domains. I then get the error: ConnectionException cURL error 28: Failed to connect to www.acme.biz port 80: Timed out Which is as far as I got when using Guzzle

I'm assuming there must be a way of working through a list of domains and grabbing their HTML without exceptions killing the process?

Mr Fett
  • 7,979
  • 5
  • 20
  • 21

1 Answers1

0

You are using Laravel 7.x.

You can also use this HTTP-CLIENT provided by Laravel. It also use Guzzle behind.

Use this facade:

use Illuminate\Support\Facades\Http;

And you can simply just write the request like this.

Http::get([URL]);

In your case you can modify this.

Http::get('http://acme.biz');

If you want a json response you can also do this:

$response = Http::get('http://acme.biz')->json();
return $response;
Japs
  • 977
  • 2
  • 10
  • 19
  • Thanks for the suggestion but it appears that because the HTTP Client is simply a Guzzle wrapper, it has exactly the same problem. – Mr Fett Jul 03 '20 at 12:36
  • you can also see this [help] (https://stackoverflow.com/questions/29822686/curl-error-60-ssl-certificate-unable-to-get-local-issuer-certificate) – Japs Jul 06 '20 at 08:54
  • Thanks, that might solve the SSL certificate if I renabled SSL verification but my problem with not being able to catch the timeout is a different issue. – Mr Fett Jul 09 '20 at 06:32
  • you can use `try {} catch($e) {}` method to ensure the response – Japs Jul 10 '20 at 01:00
  • or you can use this `set_time_limit(10000);` but I recommend not to use it – Japs Jul 10 '20 at 01:02