Regular Expressions to match an tag

Question

I am writing a small java program for a class, and I can't quite figure out why my regex isn't working properly. In the special case of having 2 tags on the same line that is read in, it only matches the second one.

Here is a link that has the regex included, along with a simple set of test data: Regex Test Link.

In my java program I have the following code:

Pattern p = Pattern.compile(regex, Pattern.CASE_INSENSITIVE);
String[] results;
System.out.println(p.toString());
Matcher m = null;
        
while((line = input.readLine()) != null) {
    m = p.matcher(line);
    while(m.find()) {
        System.out.println("Matches: " + m.group(1));
    }
}

The goal is to extract the href value, as long as it starts with http://, the website ends in either no page (like http://www.google.com) or ends in index.htm or index.html (like http://www.google.com/index.html).

My regex works for every case of the above, but doesnt match in the special case of 2 tags that are on the same line.

Any help is appreciated.

Answer 1

Just use a proper HTML parsing library, such as HTML cleaner. It is theoretically impossible to properly parse HTML with a regex - there are so many constructs that will confound it. For example:

<![CDATA[ > <a href="http://foo.com">bar</a> ]]>

This is not a link. This is literal text in XHTML.

<a href="http://bar.com/?<a href=http://foo.com>bar</a>">baz</a>

This is only one link.

<a rel="next" href="bar?2">Next</a>

This is a realistic example of a link with a relation attribute and a relative URI.

<a name="foo">The href="http://example.com" part is the link destination...</a>

This is a named anchor, not a link. However your regex would parse out the literal text here as a link.

<a
href="http://example.com">Foo</a>

Does your regex handle line-spanning links properly?

There are all kinds of other Fun edge cases that can occur. Save yourself time and headaches. These problems have already been solved and wrapped up in nice neat libraries for you to use. Take advantage of this.

Regexes may be a powerful tool, but as they say - when all you have is a hammer, everything looks like a nail. You are currently trying to hammer in a screw.

Answer 2

This worked for me in that regex tester page

<a[^>]*>[^<]*</a>

Answer 3

Regex Solution

So I was playing around and realized my issue. I adjusted my regex a bit. My main problem was at the beginning my .* was causing everything to match up until the last tag, and therefore it was really only matching once instead of twice. I made that .* lazy and it matched twice instead of once. That was the only issue. Once that regex was added to java, my loop code worked fine.

Thanks everyone that responded. While you may not have provided the answer, your comments got me thinking in the right direction!

Answer 4

You would have to look through all the matches you got per line and find which one looks like a url (like with some more regex ;))