I'm trying to retrieve the HTML for a handful of websites. I was using PHP file_get_contents which worked really quite well but was slow and was a little limited so I decided to try Guzzle. Unfortunately I'm finding it really tough to stop Guzzle throwing exceptions even using their documented exception handling. My code is really simple:
$websiteFQDN = 'http://acme.biz';
$client = new \GuzzleHttp\Client(['http_errors' => false,'verify' => false, 'connect_timeout' => 121 ]);
echo '<h3>'.$websiteFQDN.'</h3>';
try {
$res = $client->request('GET', $websiteFQDN);
echo '<p>Appears Successful</p>';
}
catch (RequestException $e){
echo '<p>Exception Caught</p>';
}
I keep getting the following error:
GuzzleHttp\Exception\ConnectException cURL error 28: Failed to connect to acme.biz port 80: Timed out (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
Any suggestions on why the above isn't catching the exception and allowing the process to continue? In my testing it loops through about 20 domains with no problems but then I get the error above and kills it.
Laravel 7.0 | Guzzle 7.0.1 | PHP 7.2.5
Someone has recommended trying the Laravel HTTP Client but unfortunately it suffers from the same problems because it uses Guzzle which in turn appears to use CUrl, all I'm doing is adding layers of abstraction. I tried it with this code:
$response = Http::withOptions(['verify' => false])->get($websiteFQDN);
if($response->successful()){
echo '<p>Appears Successful</p>';
}else{
echo '<p>Exception Caught</p>';
}
When I try using Laravel HTTP vanilla, I get the "cURL error 60: SSL certificate" error mentioned here. I add the 'verify' => false Guzzle option to the HTTP Client which passes it to Guzze (just as a I did when I used Guzzle), that solves that and it gets through more domains. I then get the error: ConnectionException cURL error 28: Failed to connect to www.acme.biz port 80: Timed out Which is as far as I got when using Guzzle
I'm assuming there must be a way of working through a list of domains and grabbing their HTML without exceptions killing the process?