Java Regex - Extract link from HTML anchor

Question

I have the following code

private String anchorRegex = "\\<\\s*?a\\s+.*?href\\s*?=\\s*?([^\\s]*?).*?\\>";
private Pattern anchorPattern = Pattern.compile(anchorRegex, Pattern.CASE_INSENSITIVE);
String content = getContentAsString();
Matcher matcher = anchorPattern.matcher(content);

while(matcher.find()) {
    System.out.println(matcher.group(1));
}

The call to getContentAsString() returns the HTML content from a web page. The problem I'm having is that the only thing that gets printed in my System.out is a space. Can anyone see what's wrong with my regex?

Regex drives me crazy sometimes.

It is a bad idea, to do this: http://stackoverflow.com/questions/590747/using-regular-expressions-to-parse-html-why-not — Christian Kuetbach, Feb 27 '12 at 20:52
He's not matching HTML generally; regex is fine for HTML in constrained use-cases. — beerbajay, Feb 27 '12 at 20:55

score 1 · Accepted Answer · answered Feb 27 '12 at 20:50

1

You need to delimit your capturing group from the following .*?. There's probably double quotes " around the href, so use those:

<\s*a\s+.*?href\s*=\s*"(\S*?)".*?>

Your regex contains:

([^\s]*?).*?

The ([^\s]*?) says to reluctantly find all non-whitespace characters and save them in a group. But the reluctant *? depends on the next part, which is .; any character. So the matching of the href aborts at the first possible chance and it is the .*? which matches the rest of the URL.

answered Feb 27 '12 at 20:50

beerbajay

19,652
6
58
75

I thought about that but the problem I found is that some sites may use single quotes in the href, while some may omit both single and double quotes. – Justin Kredible Feb 27 '12 at 20:51
Single vs. double quotes is easy; use `['"]`, but if the site is not delimiting the `href` from other attributes, they should go to hell and you should not bother with them or their data. E.g. what do you expect from this: `` ? If you really need information from horribly formatted HTML, you should use something other than a regex. – beerbajay Feb 27 '12 at 20:57

score 1 · Answer 2 · answered Feb 27 '12 at 20:52

1

The regex you should be using is this:

String anchorRegex = "(?s)<\\s*a\\s+.*?href\\s*=\\s*['\"]([^\\s>]*)['\"]";

answered Feb 27 '12 at 20:52

anubhava

761,203
64
569
643

score 0 · Answer 3 · answered Feb 27 '12 at 22:41

This should be able to pull out the href without too much trouble.
The link is in capture group 2, its expanded and assumes dot-all.
Use Java delimiters as necessary.

(?s)
<a 
  (?=\s) 
  (?:[^>"']|"[^"]*"|'[^']*')*? (?<=\s) href \s*=\s* (['"]) (.*?) \1 
  (?:".*?"|'.*?'|[^>]*?)+ 
>

or not expanded, not dot-all.

<a(?=\s)(?:[^>"']|"[^"]*"|'[^']*')*?(?<=\s)href\s*=\s*(['"])([\s\S]*?)\1(?:"[\s\S]*?"|'[\s\S]*?'|[^>]*?)+>

Java Regex - Extract link from HTML anchor

3 Answers3