php curl headers do not return from website?

Question

I use this command to get headers back from this specific page.

curl -I https://www.gearbest.com/mobile-phones/pp_1686789.html?wid=1817324

i run the above command from terminal and i get these results...

HTTP/2 200
content-type: text/html; charset=UTF-8
pragma: public
last-modified: Fri, 02 Nov 2018 18:15:13 GMT
gbcdnlang: en
access-control-allow-origin: *
access-control-allow-methods: GET, POST
ng-cache: HIT
x-edgeconnect-midmile-rtt: 36
x-edgeconnect-origin-mex-latency: 137
cache-control: public, max-age=60
expires: Fri, 02 Nov 2018 23:39:15 GMT
date: Fri, 02 Nov 2018 23:38:15 GMT
set-cookie: ORIGINDC=3;Domain=.gearbest.com;Path=/
set-cookie: ORIGINDCPC=3;Domain=.gearbest.com;Path=/
set-cookie: AKAM_CLIENTID=d51abedb38c60f8a5217ae15808769fc; expires=Mon, 31-Dec-2038 23:59:59 GMT; path=/; domain=.gearbest.com
vary: User-Agent

So far so good. But when i try to implement in php using the code below i get "URL not exists" ! What am i doing wrong? Any help appreciated.

  $url1 = "https://www.gearbest.com/mobile-phones/pp_1686789.html?wid=1817324";
  $curl1 = curl_init($url1);
  curl_setopt($curl1, CURLOPT_NOBODY, true);
  curl_setopt($curl1, CURLOPT_CONNECTTIMEOUT, 0);
  curl_setopt($curl1, CURLOPT_TIMEOUT, 5); //timeout in seconds


  $result1 = curl_exec($curl1);

  if ($result1 !== false)
  {
    $statusCode1 = curl_getinfo($curl1, CURLINFO_HTTP_CODE);
    if ($statusCode1 == 404)
    {
      echo "URL Not Exists";
    }
    else
    {
       echo "URL Exists";
    }
  }
  else
  {
    echo "URL not Exists";
  }

what you get if you type on the terminal `$ php -i | grep "SSL Version"` ? — Paolo, Nov 04 '18 at 22:30

Paolo · Answer 1 · 2018-11-07T10:07:12.550

2

First of all, the code you posted is valid. You don't need to alter it.

If you just try it with ex. https://www.google.com it works.

The problem is that your version of PHP uses LibreSSL for php-curl HTTPS connections.

I have the same issue on macOS where my version of PHP uses OpenTransport.

The website you're connecting to requires to establish a SSL connection in a way that LibreSSL is not able to handle.

You need to compile/install OpenSSL and run your code on a PHP interpreter that has been compiled to use it for php-curl.

Update:

as you're on macOS you may compile PHP from source. Googling for macos compile php from source will return a list of "recipes" to do that.

If you use brew to compile PHP (and all the "web applications stack") you may take a look here:

osx 10.10 Curl POST to HTTPS url gives SSLRead() error

The question (and answer) also address a problem similar to the one in your question.

edited Nov 07 '18 at 10:07

answered Nov 05 '18 at 18:47

Paolo

15,233
27
70
91

I am on MacOS too. So i have to install OpenSSL but then i have to find a tutorial how to do the rest of the procedure... – stefanosn Nov 06 '18 at 21:36
@stefanosn see the update on my answer. hope it helps! – Paolo Nov 07 '18 at 10:07
Thanks for the guide i appreciate it! – stefanosn Nov 08 '18 at 23:18

score 1 · Answer 2 · answered Nov 04 '18 at 23:13

1

Try adding the CURLOPT_SSL_VERIFYPEER option, it connects via SSl but checks the certificate, like this:

<?PHP
  $handle=curl_init('https://www.google.com');
  curl_setopt($handle, CURLOPT_VERBOSE, true);
  curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($handle, CURLOPT_SSL_VERIFYPEER, false);
  $content = curl_exec($handle);
  echo $content;
?>

answered Nov 04 '18 at 23:13

Rafael Maciel

9
1

i tried it but i get Error:LibreSSL SSL_read: SSL_ERROR_SYSCALL, errno 60 on this page https://www.gearbest.com/mobile-phones/pp_1686789.html?wid=1817324 . It works on https://www.gearbest.com but not in a specific product page! – stefanosn Nov 05 '18 at 13:17

score 0 · Answer 3 · answered Nov 04 '18 at 23:00

I think that is not your problem. I think they block something when it's requested over PHP. When you use file_get_contents you get the following output.

Warning: file_get_contents(): SSL: Operation timed out in /test.php on line 2
Warning: file_get_contents(https://www.gearbest.com/mobile-phones/pp_1686789.html?wid=1817324): failed to open stream: HTTP request failed!  in /test.php on line 2

When you use the following url in your script it's working fine (https://www.test.de).

I have made some other tests with your script but i get everytime a result and your excepted result.

Error:LibreSSL SSL_read: SSL_ERROR_SYSCALL, errno 60 ... i am just curious how is this possible! There has to be a way! — stefanosn, Nov 05 '18 at 13:18

php curl headers do not return from website?

3 Answers3

Linked