3

I am trying to write a RegEx rule to find all a href HTML links on my webpage and add a 'rel="nofollow"' to them.

However, I have a list of URLs that must be excluded (for exmaple, ANY (wildcards) internal link (eg. pokerdiy.com) - so that any internal link that has my domain name in is excluded from this. I want to be able to specify exact URLs in the exclude list too - for example - http://www.example.com/link.aspx)

Here is what I have so far which is not working:

(]+)(href="http://.*?(?!(pokerdiy))[^>]+>)

If you need more background/info you can see the full thread and requirements here (skip the top part to get to the meat): http://www.snapsis.com/Support/tabid/601/aff/9/aft/13117/afv/topic/afpgj/1/Default.aspx#14737

Rodney
  • 5,417
  • 7
  • 54
  • 98
  • 1
    Don't. http://www.codinghorror.com/blog/2009/11/parsing-html-the-cthulhu-way.html – TrueWill Mar 15 '10 at 22:45
  • 1
    DOES NOT COMPUTE! http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454 – dkarzon Mar 15 '10 at 22:50
  • 5
    Ok, I have read the attached and get the point. However, my requirements are limited and parsing with RegEx IS my only option. I appreciate the following: * It's generally a bad idea. (from Coding Horror). So... with that in mind - can anyone help with my RegEx code please? I can explain why I am limited to regex parsing later, but I need a solution please, no matter how much of a bad idea it may seem. – Rodney Mar 16 '10 at 01:53
  • This doesnt really apply here the href parameter is simple enough to be parsed by a regex especially when its a limited subset of all possible hrefs i.e. only the hrefs in your pages! – James Anderson May 25 '10 at 06:03

3 Answers3

10

An improvement to James' regex:

(<a\s*(?!.*\brel=)[^>]*)(href="https?://)((?!(?:(?:www\.)?'.implode('|(?:www\.)?', $follow_list).'))[^"]+)"((?!.*\brel=)[^>]*)(?:[^>]*)>

This regex will matches links NOT in the string array $follow_list. The strings don't need a leading 'www'. :) The advantage is that this regex will preserve other arguments in the tag (like target, style, title...). If a rel argument already exists in the tag, the regex will NOT match, so you can force follows on urls not in $follow_list

Replace the with: 

$1$2$3"$4 rel="nofollow">

Full example (PHP):

function dont_follow_links( $html ) {
 // follow these websites only!
 $follow_list = array(
  'google.com',
  'mypage.com',
  'otherpage.com',
 );
 return preg_replace(
  '%(<a\s*(?!.*\brel=)[^>]*)(href="https?://)((?!(?:(?:www\.)?'.implode('|(?:www\.)?', $follow_list).'))[^"]+)"((?!.*\brel=)[^>]*)(?:[^>]*)>%',
  '$1$2$3"$4 rel="nofollow">',
  $html);
}

If you want to overwrite rel no matter what, I would use a preg_replace_callback approach where in the callback the rel attribute is replaced separately:

$subject = preg_replace_callback('%(<a\s*[^>]*href="https?://(?:(?!(?:(?:www\.)?'.implode('|(?:www\.)?', $follow_list).'))[^"]+)"[^>]*)>%', function($m) {
    return preg_replace('%\srel\s*=\s*(["\'])(?:(?!\1).)*\1(\s|$)%', ' ', $m[1]).' rel="nofollow">';
}, $subject);
para
  • 684
  • 5
  • 19
  • @para what if i want to prevent user from adding his own `rel` attribute? i need to prevent following links that are not listed in `$follow_list` – alex Oct 06 '16 at 06:53
  • 1
    @alex i would probably go for a preg_replace_callback approach as the easiest solution. i updated my answer accordingly – para Dec 13 '16 at 13:48
6

I've developed a slightly more robust version that can detect whether the anchor tag already has "rel=" in it, therefore not duplicating attributes.

(<a\s*(?!.*\brel=)[^>]*)(href="https?://)((?!blog.bandit.co.nz)[^"]+)"([^>]*)>

Matches

<a href="http://google.com">Google</a>
<a title="Google" href="http://google.com">Google</a>
<a target="_blank" href="http://google.com">Google</a>
<a href="http://google.com" title="Google" target="_blank">Google</a>

But doesn't match

<a rel="nofollow" href="http://google.com">Google</a>
<a href="http://google.com" rel="nofollow">Google</a>
<a href="http://google.com" rel="nofollow" title="Google" target="_blank">Google</a>
<a href="http://google.com" title="Google" target="_blank" rel="nofollow">Google</a>
<a href="http://google.com" title="Google" rel="nofollow" target="_blank">Google</a>
<a target="_blank" href="http://blog.bandit.co.nz">Bandit</a>

Replace using

$1$2$3"$4 rel="nofollow">

Hope this helps someone!

James

James Nisbet
  • 374
  • 2
  • 13
4
(<a href="https?://)((?:(?!\b(pokerdiy.com|www\.example\.com/link\.aspx)\b)[^"])+)"

would match the first part of any link that starts with http:// or https:// and doesn't contain pokerdiy.com or www.example.com/link.aspx anywhere in the href attribute. Replace that by

\1\2" rel="nofollow"

If a rel="nofollow" is already present, you'll end up with two of these. And of course, relative links or other protocols like ftp:// etc. won't be matched at all.

Explanation:

(?!\b(foo|bar)\b)[^"] matches any non-" character unless it it possible to match foo or bar at the current location. The \bs are there to make sure we don't accidentally trigger on rebar or foonly.

This whole contruct is repeated ((?: ... )+), and whatever is matched is preserved in backreference \2.

Since the next token to be matched is a ", the entire regex fails if the attribute contains foo or bar anywhere.

Tim Pietzcker
  • 328,213
  • 58
  • 503
  • 561