allow_url_fopen is on, but doesn't work

Question

I've a problem with a fetch in my server.

I'm trying to get all content from a url, and save it in a variable.

But, this return fopen die message.

This is my code:

//Guardo la url pasada por get
$url = $_GET["url"];

if(preg_match('#^http://www.filmaffinity.com.*#s', trim($url))){
  //Funciona
} else{
$data = array('msg' => 'bad url');
   echo json_encode($data);
   return false;
}
//Tomo el código y lo meto en una variable
$fo= fopen($url,"r") or die ("No se encuentra la pagina.");
   while (!feof($fo)) {
   $cadena .= fgets($fo, 4096);
}
fclose ($fo);

This is part of my code, when i execute this, return 'No se encuentra la pagina.'

allow_url_fopen is ON in my server.

WHM PHP Config

PHP Info

How can fix this problem? Can help me?

Thanks.

Does this help you? http://stackoverflow.com/questions/24689153/php-get-fopen-http-response-error-code — O. Jones, Feb 19 '15 at 23:46

Misunderstood · Accepted Answer · 2015-02-20T02:08:05.910

You are just looking for a certain name in the url?
This will work better.

if(stripos('http://www.filmaffinity.com',$url)){}

you may want to try:

file_get_contents($url)

You may want to try encoding the url:

$url = urlencode($url)

or

$url = rawurlencode($url)

curl will get you more answers to your problem:
with curl you can control the the timeout, see the request (curl_getinfo) and response (CURLINFO_HEADER_OUT, true) headers, the HTTP Status (['http_code']), it can follow 30x redirects, etc.

There is an issue:

$data = file_get_contents('http://www.filmaffinity.com');

Returned an error:

 file_get_contents(http://www.filmaffinity.com) [<a href='function.file-get-contents'>function.file-get-contents</a>]: failed to open stream: HTTP request failed! HTTP/1.0 500 Internal Server Error

With curl:

HTTP CODE: aarray (
  'url' => 'http://www.filmaffinity.com/',
  'content_type' => 'text/html',
  'http_code' => 302,
  'header_size' => 223,
  'request_size' => 170,
  'filetime' => -1,
  'ssl_verify_result' => 0,
  'redirect_count' => 0,
  'total_time' => 0.321491,
  'namelookup_time' => 0.040338,
  'connect_time' => 0.180309,
  'pretransfer_time' => 0.180365,
  'size_upload' => 0,
  'size_download' => 20,
  'speed_download' => 62,
  'speed_upload' => 0,
  'download_content_length' => 20,
  'upload_content_length' => -1,
  'starttransfer_time' => 0.321415,
  'redirect_time' => 0,
  'certinfo' => 
  array (
  ),
  'redirect_url' => 'http://www.filmaffinity.com/en/main.html',
  'request_header' => 'GET / HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:32.0) Gecko/20100101 Firefox/32.0
Host: www.filmaffinity.com
Accept: */*
Accept-Encoding: deflate, gzip

With curl it works, but you must change this:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);

To true:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

Tested Working Code:

<?php
header('Content-Type: text/plain; charset=utf-8');
echo "start\n";
$url = 'http://www.filmaffinity.com';
 $ch = curl_init($url);
  curl_setopt($ch, CURLOPT_ENCODING,"");
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_HEADER, true);
  curl_setopt($ch, CURLINFO_HEADER_OUT, true);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($ch, CURLOPT_FILETIME, true);
  curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (Windows NT 5.1; rv:32.0) Gecko/20100101 Firefox/32.0");
  curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 100);
  curl_setopt($ch, CURLOPT_VERBOSE, true);
  curl_setopt($ch, CURLOPT_AUTOREFERER, true);
  curl_setopt($ch, CURLOPT_TIMEOUT,100);
  curl_setopt($ch, CURLOPT_FAILONERROR,true);
  $data = curl_exec($ch);
  if (curl_errno($ch)){
      $data .= 'Retreive Base Page Error: ' . curl_error($ch);
  }
  else {
    $skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE)); 
    $responseHeader = substr($data,0,$skip);
    $data= substr($data,$skip);
    $info = curl_getinfo($ch);
    if ($info['http_code'] != '200')
    $info = var_export($info,true);
   }
  if ($info['http_code'] != '200'){echo 'HTTP CODE: ' .$info['http_code'];}


preg_match_all('#<a href="([^"]*)#m',$data,$matches);
foreach($matches[1] as $val){
  $url = 'http://www.filmaffinity.com' . $val;
  echo "$url\n";
}
?>

This is the Result

 http://www.filmaffinity.com/en/main.html
    http://www.filmaffinity.com/en/advsearch.php
    http://www.filmaffinity.com/en/login.php
    http://www.filmaffinity.com/en/register.php
    http://www.filmaffinity.com/en/main.html
    http://www.filmaffinity.com/en/awards.php?award_id=berlin&year=2015
    http://www.filmaffinity.com/en/awards.php?award_id=academy_awards&year=2015
    http://www.filmaffinity.com/en/cat_new_th_us.html
    http://www.filmaffinity.com/en/boxoffice.php
    http://www.filmaffinity.com/en/imlost.php
    http://www.filmaffinity.com/en/all_awards.php
    http://www.filmaffinity.com/en/best_2014.php
    http://www.filmaffinity.com/en/oscar_data.php
    http://www.filmaffinity.com/en/topgen.php?nodoc=1
    http://www.filmaffinity.comhttp://www.filmaffinity.com/es/main.html
    http://www.filmaffinity.com/en/cookies_info.php
    http://www.filmaffinity.com/en/tours.php
    http://www.filmaffinity.com/en/tour.php?idtour=55
    http://www.filmaffinity.com/en/tour.php?idtour=54
    http://www.filmaffinity.com/en/tour.php?idtour=29
    http://www.filmaffinity.com/en/tour.php?idtour=9
    http://www.filmaffinity.com/en/tour.php?idtour=24
    http://www.filmaffinity.com/en/tours.php
    http://www.filmaffinity.com/en/trailers.php
    http://www.filmaffinity.com/en/bestrated.php
    http://www.filmaffinity.com/en/film489970.html
    http://www.filmaffinity.com/en/film221477.html
    http://www.filmaffinity.com/en/film730528.html
    http://www.filmaffinity.com/en/film809297.html
    http://www.filmaffinity.com/en/film399474.html
    http://www.filmaffinity.com/en/film795770.html
    http://www.filmaffinity.com/en/film536488.html
    http://www.filmaffinity.com/en/film695552.html
    http://www.filmaffinity.com/en/bestrated.php
    http://www.filmaffinity.com/en/mostrated.php
    http://www.filmaffinity.com/en/film575568.html
    http://www.filmaffinity.com/en/film670216.html
    http://www.filmaffinity.com/en/film124904.html
    http://www.filmaffinity.com/en/film636539.html
    http://www.filmaffinity.com/en/film206955.html
    http://www.filmaffinity.com/en/film779937.html
    http://www.filmaffinity.com/en/film617730.html
    http://www.filmaffinity.com/en/film423489.html
    http://www.filmaffinity.com/en/mostrated.php
    http://www.filmaffinity.com/en/worstrated.php
    http://www.filmaffinity.com/en/film189979.html
    http://www.filmaffinity.com/en/film612348.html
    http://www.filmaffinity.com/en/film968394.html
    http://www.filmaffinity.com/en/film651247.html
    http://www.filmaffinity.com/en/film281762.html
    http://www.filmaffinity.com/en/film886013.html
    http://www.filmaffinity.com/en/film591128.html
    http://www.filmaffinity.com/en/film381051.html
    http://www.filmaffinity.com/en/worstrated.php
    http://www.filmaffinity.com/en/mostvisited.php
    http://www.filmaffinity.com/en/film124904.html
    http://www.filmaffinity.com/en/film994565.html
    http://www.filmaffinity.com/en/film941942.html
    http://www.filmaffinity.com/en/film575568.html
    http://www.filmaffinity.com/en/film670216.html
    http://www.filmaffinity.com/en/film423489.html
    http://www.filmaffinity.com/en/film809035.html
    http://www.filmaffinity.com/en/film391687.html
    http://www.filmaffinity.com/en/film934433.html
    http://www.filmaffinity.com/en/mostvisited.php
    http://www.filmaffinity.com/en/cat_new_th_us.html
    http://www.filmaffinity.com/en/film124904.html
    http://www.filmaffinity.com/en/film376816.html
    http://www.filmaffinity.com/en/film898006.html
    http://www.filmaffinity.com/en/film562434.html
    http://www.filmaffinity.com/en/film510733.html
    http://www.filmaffinity.com/en/film272576.html
    http://www.filmaffinity.com/en/film493854.html
    http://www.filmaffinity.com/en/film792317.html
    http://www.filmaffinity.com/en/film784978.html
    http://www.filmaffinity.com/en/cat_new_th_us.html
    http://www.filmaffinity.com/en/cat_upc_th_us.html
    http://www.filmaffinity.com/en/film526524.html
    http://www.filmaffinity.com/en/film405261.html
    http://www.filmaffinity.com/en/film543207.html
    http://www.filmaffinity.com/en/film809035.html
    http://www.filmaffinity.com/en/film402986.html
    http://www.filmaffinity.com/en/film956269.html
    http://www.filmaffinity.com/en/film759419.html
    http://www.filmaffinity.com/en/film699453.html
    http://www.filmaffinity.com/en/film701069.html
    http://www.filmaffinity.com/en/cat_upc_th_us.html
    http://www.filmaffinity.comhttp://www.facebook.com/FilmAffinity
    http://www.filmaffinity.comhttp://twitter.com/Filmaffinity
    http://www.filmaffinity.com/en/faq.php
    http://www.filmaffinity.com/en/private.php

Then Add (replace the foreach loop above):

foreach($matches[1] as $key => $val){
  if (!strpos($val,'/film')){continue;}
  $url = 'http://www.filmaffinity.com' . $val;
  $ch = curl_init($url);
  curl_setopt($ch, CURLOPT_ENCODING,"");
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_HEADER, true);
  curl_setopt($ch, CURLINFO_HEADER_OUT, true);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($ch, CURLOPT_FILETIME, true);
  curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (Windows NT 5.1; rv:32.0) Gecko/20100101 Firefox/32.0");
  curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 100);
  curl_setopt($ch, CURLOPT_VERBOSE, true);
  curl_setopt($ch, CURLOPT_AUTOREFERER, true);
  curl_setopt($ch, CURLOPT_TIMEOUT,100);
  curl_setopt($ch, CURLOPT_FAILONERROR,true);
  $data = curl_exec($ch);
  if (curl_errno($ch)){
      $data .= 'Retreive Base Page Error: ' . curl_error($ch);
  }
  else {
    $skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE)); 
    $responseHeader = substr($data,0,$skip);
    $data= substr($data,$skip);
    $info = curl_getinfo($ch);
    if ($info['http_code'] != '200')
    $info = var_export($info,true);
   }
  if ($info['http_code'] != '200'){echo 'HTTP CODE: ' .$info['http_code'];}
  $fp = fopen("html$key.txt",'w');
  fwrite($fp,$data);
  fclose($fp);
}

I'm trying to get all content from Filmaffinity, save in a variable, and print this with jQuery in Wordpress inputs. — Ferrmolina, Feb 20 '15 at 00:03
are you scraping the page to get the correct url? Like: `$data = file_get_contents('http://www.filmaffinity.com'); preg_match_all('# — Misunderstood, Feb 20 '15 at 00:30
No no, im scraping the page, to get the content of this. See all code: http://pastebin.com/ThCfyN5i — Ferrmolina, Feb 20 '15 at 00:32
There is a 302 Redirect. See end of my post: `'http_code' => 302,` — Misunderstood, Feb 20 '15 at 00:44
It is now working perfectly. Copy and paste my updated post. — Misunderstood, Feb 20 '15 at 01:00
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/71300/discussion-between-misunderstood-and-ferrrmolina). — Misunderstood, Feb 20 '15 at 01:04

score 0 · Answer 2 · answered Feb 19 '15 at 23:48

0

Use http://php.net/manual/en/function.file-get-contents.php instead of fopen for this.

$fo = file_get_contents($url);

answered Feb 19 '15 at 23:48

score 0 · Answer 3 · answered Feb 19 '15 at 23:49

0

The allow_url_fopen method fails if the web service does not return a successful response. When sending a curl request to your http://www.filmaffinity.com URL, I see a HTTP error, I have seen both a 500, and 403. If you respond with a 200 OK, then you'll not have this issue.

<?php

//Guardo la url pasada por get
$url = "http://www.google.com";

//Tomo el código y lo meto en una variable
$fo= fopen($url,"r") or die ("No se encuentra la pagina.");
while (!feof($fo)) {
  $cadena .= fgets($fo, 4096);
}

echo($cadena);
fclose ($fo);

?>

answered Feb 19 '15 at 23:49

okanaganrusty

1
1

In XAMP localhost, i get 500 server error, but in my webhosting this return `no se encuentra la pagina`, how to respond with a 200? sorry my english is poor – Ferrmolina Feb 19 '15 at 23:56
You cannot respond with a 200. The 200 is an HTTP response code. You can grab the redirect location and get the redirect but curl is easier with the `FOLLOWLOCATION`, but sometimes you need to use the cookies for the next request. – Misunderstood Feb 20 '15 at 02:00

O. Jones · Answer 4 · 2015-02-19T23:56:31.793

0

You may wish to try this, to display what went wrong with your fopen() attempt. The HTTP response header should help you understand.

$fo= @fopen($url,"r"); 
if ( !$fo ) {
  echo "No se encuentra la pagina: ".  $http_response_header[0];
}
else {
   while (!feof($fo)) {
       $cadena .= fgets($fo, 4096);
   }
}
fclose ($fo);

edited Feb 19 '15 at 23:56

answered Feb 19 '15 at 23:51

O. Jones

103,626
17
118
172

The page don't stop loading, see an example: http://peliculas.enlatino.net/comprobar.php?url=http://www.filmaffinity.com/film913067.html – Ferrmolina Feb 20 '15 at 00:00
`http://www.filmaffinity.com/film913067.html` creates a 404 page Not Found and your fopen must not have a timeout. With curl you can set a time out in seconds. – Misunderstood Feb 20 '15 at 00:07
Sorry about that, the real url is: `http://www.filmaffinity.com/es/film913067.html` and get `No se encuentra la pagina: HTTP/1.0 500 Internal Server Error` – Ferrmolina Feb 20 '15 at 00:09
The web app at `filmaffinity.com` is crashing as it tries to respond to the URL you sent it. That's what `500` means. But it worked in my Chrome browser. That server may be set up to reject requests from programs. Still, `500` is a crude kind of rejection. Your program seems to be working correctly (as long as the URL is correct). You may want to talk to the people at `filmaffinity.com` to find out how to access their site. – O. Jones Feb 20 '15 at 00:22

allow_url_fopen is on, but doesn't work

4 Answers4

Tested Working Code:

This is the Result