file_get_contents returning error 500 when page exists

Question

I am trying to scrape some data from this webpage.

I have begun by using this code:

$list_url = "http://www.jackwills.com/en-GB/ladies/sale";
$html = file_get_contents($list_url);
echo $html;

However, it's returning the website's error 500 despite the fact that file_get_contents() works with other domains.

Is there a workaround that lets me grab the page I'm after?

UPDATE

Setting ini_set('user_agent','Mozilla/4.0 (compatible; MSIE 6.0)'); has got me halfway there and I now see an HTML page with no styling. The information I need is generated by JavaScript - is there any way to get that?

Is your configuration set to allow you to open URLs? http://ca2.php.net/manual/en/filesystem.configuration.php#ini.allow-url-fopen — faintsignal, Mar 29 '14 at 15:55
@AmalMurali looking at that link it doesn't work there - that's the error page (though my mistake, it's 500 not 404). [This](http://www.jackwills.com/en-GB/ladies/sale) is the page I'm expecting to see. Unless even within that page it's working differently for me! — Sebastian, Mar 29 '14 at 16:03
Same problem/solution? http://stackoverflow.com/questions/10524748/php-file-get-contents-500-internal-server-error — faintsignal, Mar 29 '14 at 16:06
@Sebastian So did the workaround in the solution of the question I linked to not work for you? — faintsignal, Mar 29 '14 at 16:58
It gets me halfway there! The HTML for the right page loads, but no JavaScript which seems to be generating most of the content that I need. — Sebastian, Mar 29 '14 at 17:07

score 0 · Answer 1 · answered Mar 29 '14 at 16:06

0

It works for me. check your php.ini related setting such as "allow_url_fopen" that must be on:

allow_url_fopen = On

answered Mar 29 '14 at 16:06

Thanks for your reply. What you have there is the contents of my `php.ini` file. – Sebastian Mar 29 '14 at 16:09

file_get_contents returning error 500 when page exists

1 Answers1