4

I do not know why, but I have one specific URL that does not timeout (tested several thousand others without problems):

<?php
$url = 'https://....';
$context = stream_context_create(array(
    'http' => array(
        'follow_location' => false,
        'timeout' => 2,
    )
));
file_get_contents($url, false, $context);
?>

If I set follow_location to true it works without problems.

Update 1
The call of file_get_contents does not respect the 'timeout' => 2 and it does not respect PHP max_execution_time, but there seems to be a third timeout on the server itself. It returns an Internal Server Error 500 after 10 (!) minutes.

Update 2
I tried the same URL with cURL and there is no timeout problem but it returns false:

<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$contents = curl_exec($ch);
var_dump($contents);
?>

What is needed to let cURL return false?

Update 3
Ok now I catched it:

if ($contents === false) {
    echo curl_error($ch) . ' (' . curl_errno($ch) . ')' . PHP_EOL;
}

returns:

SSL certificate problem: unable to get local issuer certificate (60)

This means every time this error occurs file_get_contents() does not respect an timeout if follow_location is set to false. Sounds like a bug for me.

Update 4
As @huggilou suggested I tried several additional settings and after setting this I'm able to load the URL:

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);

Is there a similar setting for file_get_contents()?

Update 5

Now I tried the following:

$context = stream_context_create(array(
    'http' => array(
        'follow_location' => false,
        'timeout' => 2,
    ),
    'ssl' => array(
        'verify_peer' => false,
    ),
));
echo file_get_contents($url, false, $context);

This caused the timeout problem. After that I changed verify_peer to true and got this result:


Warning: file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed in /usr/www/users//t051.php on line 12

Warning: file_get_contents(): Failed to enable crypto in /usr/www/users//t051.php on line 12

Warning: file_get_contents(https://...): failed to open stream: operation failed in /usr/www/users//t051.php on line 12

By that we know: verfiy_peer is set to false by default.

And to remind: If I set verify_peer to false and follow_location to true the website is loaded!?

Update 6
As @huggilou pointed to that:

<?php
$w = stream_get_wrappers();
echo 'allow_url_fopen: ', ini_get('allow_url_fopen') ? 'yes':'no', PHP_EOL;
echo 'openssl: ',  extension_loaded('openssl') ? 'yes':'no', PHP_EOL;
echo 'http wrapper: ', in_array('http', $w) ? 'yes':'no', PHP_EOL;
echo 'https wrapper: ', in_array('https', $w) ? 'yes':'no', PHP_EOL;
echo 'wrappers: ', var_dump($w);
?>

returns:

allow_url_fopen: yes
openssl: yes
http wrapper: yes
https wrapper: yes
wrappers: array(12) {
  [0]=>
  string(5) "https"
  [1]=>
  string(4) "ftps"
  [2]=>
  string(13) "compress.zlib"
  [3]=>
  string(14) "compress.bzip2"
  [4]=>
  string(3) "php"
  [5]=>
  string(4) "file"
  [6]=>
  string(4) "glob"
  [7]=>
  string(4) "data"
  [8]=>
  string(4) "http"
  [9]=>
  string(3) "ftp"
  [10]=>
  string(4) "phar"
  [11]=>
  string(3) "zip"
}

Update 7
cURL returns for https://www.example.com/foo.html this header:

string(138) "HTTP/1.1 301 Moved Permanently
Location: http://example.com/foo.html
Content-Length: 0
Content-Type: text/html; charset=UTF-8

"

Update 8
https://www.ssllabs.com/ssltest/analyze.html?d=example.com returns: enter image description here

Update 9
I tried the same script and URL through a foreign server and there it works. Of course it needs verify_peer set to false to avoid the ssl certificate error posted in Update 5. I checked phpinfo(). There are some differences. But both use SSL Version OpenSSL/1.0.1e and PHP Version 5.5.23 through CGI/FastCGI, but one setting on my server attracts my attention:

Registered Stream Socket Transports tcp, udp, unix, udg, ssl, sslv3, tls

The foreign server has in addition sslv2. Could that be the reason?

Update 10 - temporarily solved
After exchanging some emails with my hosting provider a technician wanted to play around with a new script (follow_location=false and verify_peer=false) and executed it through the console:

php -d allow_url_fopen=On ./t052.php |less

Then he emailed me that he had no problem. I tested it by myself through the browser and suprisingly his and all my other test scripts work again?! He confirmed me that he did not change anything. This is really frustrating as it looks like a random behaviour now. As this project is still running and collecting URLs I will update my question if it happens again.

mgutt
  • 5,867
  • 2
  • 50
  • 77
  • I tested this and it loads in under 2 seconds which is why it's not timing out for me. – fire Apr 14 '15 at 13:29
  • ok you should close this question then – fire Apr 15 '15 at 08:40
  • After I know what the problem is I will do that or report updates. – mgutt Apr 15 '15 at 08:52
  • There is a post in SO about [`file_get_contents` with SSL](http://stackoverflow.com/questions/1975461/how-to-get-file-get-contents-work-with-https) – jmgross Apr 15 '15 at 09:20
  • Regarding your **Update 9** ... do I understand correctly that using PHP 5.5.23 with `"verify_peer" => false` gives you the output you expect? –  Apr 15 '15 at 13:42
  • @rdlowrey Yes. This is needed because the SSL certificate is not valid at all. This means the same script worked on the foreign server but with no user on my own. I tested three different users on this server with three different PHP versions. And now see **Update 10** :( – mgutt Apr 15 '15 at 14:44
  • 2
    @mgutt Well it's not technically "random" behavior :) PHP's encrypted https streams *never* (until very recently) observed socket timeout settings. I know this because I've specifically been working to fix this issue. It's a "won't fix" in PHP 5.4 as that branch is now only receiving security fixes. However, as of 5.5.23 and 5.6.7 timeouts should be honored when reading on crypto streams. If you encounter unexpected behavior in versions >= 5.5.23 or 5.6.7 please create a new bug report on bugs.php.net –  Apr 15 '15 at 15:29

2 Answers2

2

Maybe you should disable the SSL verification in your cURL request :

curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);

And set timeout too :

curl_setopt($ch, CURLOPT_CONNECTTIMEOUT ,30); 
curl_setopt($ch, CURLOPT_TIMEOUT, 400);

Update :

For file_get_contents, according to this answer, the php_openssl extension must be enabled and allow_url_fopen have to be active

Community
  • 1
  • 1
jmgross
  • 2,306
  • 1
  • 20
  • 24
  • I updated my answer with the solution for `file_get_contents` – jmgross Apr 15 '15 at 09:28
  • Regarding your update: As I said in my question I tested several thousand URLs before without problems. In **Update 6** you can see that all needed features are available. – mgutt Apr 15 '15 at 09:40
  • So it seems that the certificate of your url isn't valid. Please open your url in a browser to check his validity, the SSL version,... – jmgross Apr 15 '15 at 09:49
  • My browser has no problem with this URL. @fire tested the URL on his own server and he had not problem. So it must be something related to my server setup. But this does not answer why the timeouts are not respected. And it does not answer why this happens only if `follow_location` is set to `false` – mgutt Apr 15 '15 at 10:02
  • Maybe your securised URL redirect to a non secure URL. Do you have some entries in the Network log of your browser ? – jmgross Apr 15 '15 at 10:09
  • As `follow_location` is set to `false` the redirect can not be the problem. Only for your interest: It redirects from https://www.example.com to http://example.com so the target is not unsecure as its not encrypted. Again you will find some updates in my question. – mgutt Apr 15 '15 at 11:00
  • Maybe take a look to [this answer](http://stackoverflow.com/a/23130512/4682796). I hope it will solve your problem – jmgross Apr 15 '15 at 11:19
0

could you try this snippet and let me know your comments,

function send($url) {
$command = 'curl -k '.$url;
return exec($command, $output, $retValue);
}
Common Man
  • 103
  • 2
  • 13