PHP Get Contents of a URL or Page

Question

I am trying to create a PHP script which can request data, such as HTML content, from an external server, then do something with the received content. Here is a generalized example of what I am trying to accomplish:

//Get the HTML generated by http://api.somesite.com/

//Now tack on the Unix timestamp of when the data was received
$myFetchedData = $dataFromExternalServer . "\n Data received at: ". time();

echo $myFetchedData;

I'm thinking I should use curl in here somewhere, but I am not sure after that. Could someone post a generalized example of how I could do this?

possible duplicate of [How do I get the HTML code of a web page in PHP?](http://stackoverflow.com/questions/819182/how-do-i-get-the-html-code-of-a-web-page-in-php) — Cees Timmerman, Oct 02 '14 at 16:44

score 36 · Accepted Answer · answered May 11 '11 at 22:41

36

If you only need GET and allow_url_fopen is enabled on your server, you can simply use

$data = file_get_contents('http://api.somesite.com');

answered May 11 '11 at 22:41

ThiefMaster

310,957
84
592
636

Yes, I just need a simple GET request. I know allow_url_fopen isn't enabled by default on some web-hosts, especially on the budget hosts. To allow maximum compatibility, which do you recommend, `file_get_contents()` or the curl library? – Oliver Spryn May 11 '11 at 22:43
Curl might also be missing. So you probably want to use whatever's available. – ThiefMaster May 11 '11 at 22:44
between the two, if ur given the choice, file_get_contents is actually the better method – Ascherer May 11 '11 at 22:53
@Ascherer Actually, I would think cURL is better. It seems to get better performance. [http://stackoverflow.com/questions/555523/file-get-contents-vs-curl-what-has-better-performance](http://stackoverflow.com/questions/555523/file-get-contents-vs-curl-what-has-better-performance). – tyronegcarter May 11 '11 at 23:08
for speed yeah, but not simplicity – Ascherer May 11 '11 at 23:14
Given the choice, I would pick cURL. You get a lot more control (should you ever need to use it) and it has more robust error handling. If you're GET'ing as part of loading your page you need to make sure you deal with what happens if that GET takes a very long time or fails. – Eli May 12 '11 at 00:05
`file_get_contents` is disabled on many servers because a lot of malicious attacks occur through it. Far better option is to use the cURL method suggested in the next answer – David Jul 13 '17 at 23:00
Ladies and Gentlemen, the truth is only the CURL option works as it should. Using file_get_contents('URL') to fetch JSON open data from external URL is very tricky. On daytime, I get timeouts and errors, once every two or three attempts. At night it works like magic - but unpredictible magic. – TomeeNS Feb 19 '19 at 10:29

score 10 · Answer 2 · answered May 11 '11 at 22:49

This is how you would use cURL to get contents from a remote url. You would define the function and make calls like url_get_contents("http://example.com/");

function url_get_contents($url, $useragent='cURL', $headers=false, $follow_redirects=true, $debug=false) {

    // initialise the CURL library
    $ch = curl_init();

    // specify the URL to be retrieved
    curl_setopt($ch, CURLOPT_URL,$url);

    // we want to get the contents of the URL and store it in a variable
    curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);

    // specify the useragent: this is a required courtesy to site owners
    curl_setopt($ch, CURLOPT_USERAGENT, $useragent);

    // ignore SSL errors
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

    // return headers as requested
    if ($headers==true){
        curl_setopt($ch, CURLOPT_HEADER,1);
    }

    // only return headers
    if ($headers=='headers only') {
        curl_setopt($ch, CURLOPT_NOBODY ,1);
    }

    // follow redirects - note this is disabled by default in most PHP installs from 4.4.4 up
    if ($follow_redirects==true) {
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); 
    }

    // if debugging, return an array with CURL's debug info and the URL contents
    if ($debug==true) {
        $result['contents']=curl_exec($ch);
        $result['info']=curl_getinfo($ch);
    }

    // otherwise just return the contents as a variable
    else $result=curl_exec($ch);

    // free resources
    curl_close($ch);

    // send back the data
    return $result;
}

T.Todua · Answer 3 · 2018-11-29T14:19:52.357

6

simple methods

<?php
echo readfile("http://example.com/");   //needs "Allow_url_include" enabled
//OR
echo include("http://example.com/");    //needs "Allow_url_include" enabled
//OR
echo file_get_contents("http://example.com/");
//OR
echo stream_get_contents(fopen('http://example.com/', "rb")); //you may use "r" instead of "rb"  //needs "Allow_url_fopen" enabled
?>

The best Way (using cURL):

echo get_remote_data('http://example.com');   //SIMPLE REQUEST;
//OR
echo get_remote_data('http://example.com', "var2=something&var3=blabla" ); //POST REQUEST;

(CODE: at GitHub )

edited Nov 29 '18 at 14:19

answered Apr 03 '13 at 12:19

T.Todua

53,146
19
236
237

1

Please don't make these kinds of edits, @solutioner. It would be preferable to flag this post as a duplicate. – Lix Nov 27 '14 at 15:44
An answer consisting solely of a link is not considered a good answer for [so]. – Lix Nov 27 '14 at 15:44
Per http://php.net/manual/en/function.include.php, it doesn't look like echoing the return from an included file will do what you're implying it will do. – Beachhouse Mar 06 '15 at 22:37

score 6 · Answer 4 · answered May 11 '11 at 22:41

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.url.com/cakephp/controller/action/param:1" ); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1 ); 
$dataFromExternalServer=curl_exec($ch);

See also: http://php.net/manual/en/function.curl-exec.php

score 1 · Answer 5 · answered May 11 '11 at 22:41

1

Put simply:

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://api.somesite.com/');
$dataFromExternalServer = curl_exec($ch);

answered May 11 '11 at 22:41

Datajam

4,141
2
23
25

1

You need to `curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1 );` or the returned data is sent to the client. – ThiefMaster May 11 '11 at 22:45
Yes of course, as well as a plethora of other curl options that can be set. I did say "put simply". – Datajam May 11 '11 at 22:45

score 0 · Answer 6 · answered May 11 '11 at 22:43

0

If your PHP installation doesn't support curl and does not allow_url_fopen, here's an option if you have PECL:

$body = http_parse_message(http_get($url))->body;

answered May 11 '11 at 22:43

Håvard S

23,244
8
61
72

2

`(PECL pecl_http >= 0.1.0)` - even more unlikely than curl or `allow_url_fopen` – ThiefMaster May 11 '11 at 22:44
My hosts have PECL, but I agree, it may be more of a long shot. – Håvard S May 11 '11 at 22:47
I agree, I was hoping to obtain maximum compatibility. Thanks though, Håvard! – Oliver Spryn May 11 '11 at 22:47

PHP Get Contents of a URL or Page

6 Answers6