PHP script that counts the number of outgoing links on a page and ignores the rel="nofollow" ones

Question

I am a php newb but I am pretty sure this will be hard to accomplish and very server consuming. But I want to ask, get the opinion of much smarter users than myself.

Here is what I am trying to do:

I have a list of URL's, an array of URL's actually.

For each URL, I want to count the outgoing links - which DO NOT HAVE REL="nofollow" attribute - on that page.

So in a way, I'm afraid I'll have to make php load the page and preg match using regular expressions all the links?

Would this work if I'd had lets say 1000 links?

Here is what I am thinking, putting it in code:

$homepage = file_get_contents('http://www.site.com/');

$homepage = htmlentities($homepage);

// Do a preg_match for http:// and count the number of appearances:
$urls = preg_match();

// Do a preg_match for rel="nofollow" and count the nr of appearances:
$nofollow = preg_match();


// Do a preg_match for the number of "domain.com" appearances so we can subtract the website's internal links:
$internal_links = preg_match();

// Substract and get the final result: 
$result = $urls - $nofollow - $internal_links;

Hope you can help, and if the idea is right maybe you can help me with the preg_match functions.

Answer 1

You can use PHP's DOMDocument class to parse the HTML and parse_url to parse the URLs:

$url = 'http://stackoverflow.com/';
$pUrl = parse_url($url);

// Load the HTML into a DOMDocument
$doc = new DOMDocument;
@$doc->loadHTMLFile($url);

// Look for all the 'a' elements
$links = $doc->getElementsByTagName('a');

$numLinks = 0;
foreach ($links as $link) {

    // Exclude if not a link or has 'nofollow'
    preg_match_all('/\S+/', strtolower($link->getAttribute('rel')), $rel);
    if (!$link->hasAttribute('href') || in_array('nofollow', $rel[0])) {
        continue;
    }

    // Exclude if internal link
    $href = $link->getAttribute('href');

    if (substr($href, 0, 2) === '//') {
        // Deal with protocol relative URLs as found on Wikipedia
        $href = $pUrl['scheme'] . ':' . $href;
    }

    $pHref = @parse_url($href);
    if (!$pHref || !isset($pHref['host']) ||
        strtolower($pHref['host']) === strtolower($pUrl['host'])
    ) {
        continue;
    }

    // Increment counter otherwise
    echo 'URL: ' . $link->getAttribute('href') . "\n";
    $numLinks++;

}

echo "Count: $numLinks\n";

Answer 2

You can use SimpleHTMLDOM:

// Create DOM from URL or file
$html = file_get_html('http://www.site.com/');

// Find all links 
foreach($html->find('a[href][rel!=nofollow]') as $element) {
    echo $element->href . '<br>';
}

As I'm not sure that SimpleHTMLDOM supports a :not selector and [rel!=nofollow] might only return a tags with a rel attribute present (and not ones where it isn't present), you may have to:

foreach($html->find('a[href][!rel][rel!=nofollow]') as $element)

Note the added [!rel]. Or, do it manually instead of with a CSS attribute selector:

// Find all links 
foreach($html->find('a[href]') as $element) {
    if (strtolower($element->rel) != 'nofollow') {
        echo $element->href . '<br>';
    }
}