PHP file_get_contents() and query string

Question

I have a problem - a parser that does not parse. It does not work! It gives not back anything! Well, I want to get something back - and store the results in a mysql-database.

<?PHP
// Original PHP code by Chirp Internet: http://www.chirp.com.au
// Please acknowledge use of this code by including this header.

$url = "http://www.edi.admin.ch/esv/00475/00698/index.html?lang=de";

//$input = @file_get_contents($url) or die("Could not access file: $url");

$input = file_get_contents($url) or die("Could not access file: $url"); 

$regexp = "<a\s[^>]*href=(\"??)([^\" >]*?)\\1[^>]*>(.*)<\/a>";
if(preg_match_all("/$regexp/siU", $input, $matches, PREG_SET_ORDER))
{
    foreach($matches as $match) 
    {
        // $match[2] = all the data i want to collect...
        // $match[3] = text that i need to collect - see a detail-page
    }
}
?>

It goes a bit over my head: It does not give back any results. Do I have to use file_get_contents() with a query string?

Are you sure you get something back from the url using file_get_contents? — Bas Slagter, May 24 '11 at 21:06
Please say what it does instead. Not working is not a question. — Amir Raminfar, May 24 '11 at 21:07
Refer to [this](http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454) answer for parsing HTML with regular expressions. — rid, May 24 '11 at 21:16
Do you actually know what "query string" means? Also why the regression (regex extracting instead of actual dom parsing) from your previous questions on the very same topic? — mario, May 24 '11 at 21:18
You assume `file_get_contents()` can access remote files, when in fact, it does not always do this. It depends on a specific setting which is turned off most of the time for security reasons. — Christian, May 24 '11 at 21:45

score 4 · Answer 1 · answered May 24 '11 at 21:11

Works fine here:

$url = "http://www.edi.admin.ch/esv/00475/00698/index.html?lang=de";

$doc = new DOMDocument();
// Supress warnings for screwy HTML
@$doc->loadHTMLFile($url);

// Use DOM functionality to get all links
$link_list = $doc->getElementsByTagName('a');

$links = array(); 
foreach($link_list as $link) {
  if($link->getAttribute('href')) {
    // and put their href attributes and
    // text content in an array
    $link_info['href'] = $link->getAttribute('href');
    $link_info['text'] = $link->nodeValue;
    $links[] = $link_info;
  }
}

print_r($links);

Output:

Array
(
    [0] => Array
        (
            [href] => #webNavigationDiv
            [text] => Direkt zur Navigation [Alt + 1]
        )

    [1] => Array
        (
            [href] => #contentStart
            [text] => Direkt zum Inhalt [Alt + 2] 
        )

    [2] => Array
        (
            [href] => #keywords_fast
            [text] => Direkt zur Suche [Alt + 5]
        )

score 3 · Answer 2 · edited May 24 '11 at 21:39

You're doing something that you shouldn't – parsing HTML with regex. Don't do it!

Use DOM parsing functions instead. PHP's DOMDocument class is quite easy to use, and much more legible (and stable) than regex:

$dom = new DOMDocument;
$dom->loadHTML($yourHTML);

$links = $dom->getElementsByTagName('a');

$hrefs = array();
foreach ($links as $link) {
    $hrefs[] = $link->getAttribute('href');
}

Getting other data, such as the text content or other attribute names, is trivially easy if you want to do so.

score 2 · Accepted Answer · answered May 24 '11 at 21:45

You can only use fopen-like functions with a url if the appropriate fopen wrapper is enabled.

See: http://www.php.net/manual/en/filesystem.configuration.php#ini.allow-url-fopen

While I would second the 'regex isn't good for html,' if this is just for a little script, who cares? That being said, DOMDocument and friends are easy enough to use.

Josh

PHP file_get_contents() and query string

3 Answers3