1

Edit: To be clear, please understand that I am not using Regex to parse the html, that's crazy talk! I'm simply wanting to clean up a messy string of html so it will parse

Edit #2: I should also point out that the control character I'm using is a special unicode character - it's not something that would ever be used in a proper tag under any normal circumstances

Suppose I have a string of html that contains a bunch of control characters and I want to remove the control characters from inside tags only, leaving the characters outside the tags alone.

For example

Here the control character is the numeral "1".

Input

The quick 1<strong>orange</strong> lemming <sp11a1n 1class1='jumpe111r'11>jumps over</span> 1the idle 1frog

Desired Output

The quick 1<strong>orange</strong> lemming <span class='jumper'>jumps over</span> 1the idle 1frog

So far I can match tags which contain the control character but I can't remove them in one regex. I guess I could perform another regex on my matches, but I'd really like to know if there's a better way.

My regex

Bear in mind this one only matches tags which contain the control character.

<(([^>])*?`([^>])*?)*?>

Thanks very much for your time and consideration.

Iain Fraser

Iain Fraser
  • 6,578
  • 8
  • 43
  • 68
  • 1
    To state the obvious once more.... http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454 – Robin Day May 12 '10 at 08:04
  • 1
    Yes, yes, yes, I know, I know. But I'm not PARSING the html. People immediately jump to conclusions when they see html and regex in the same sentence. In this instance it just so happens that I'm working with html, but this problem could apply to any kind of string: "I want to remove all instances of the '!' character that fall between curly braces". This is the same class of problem - what the string actually means is irrelevant. – Iain Fraser May 12 '10 at 08:12
  • And for the record, the unprocessed strings may not even parse as html (if the control character falls within the tag name for example). This is why I want to remove them from inside of tags. – Iain Fraser May 12 '10 at 08:14
  • As a suggestion, I would alter your question to describe the problem you're having. Not to dictate the way you wish to do it. There is no reason to assume that a regex is the solution you need. If you have no mention of regex's other than showing your attempt at a solution then people can look at the problem from a different angle. – Robin Day May 12 '10 at 08:34
  • Noted. I know html/regex is a touchy subject and yeah, I guess just because I decided to use regex doesn't mean someone else would. I just wanted a solution I could write in one line and that's why regex struck me as the right tool – Iain Fraser May 13 '10 at 00:22

3 Answers3

2

Regex isn't the tool for this, but you can use lookbehind and lookahead to match 1 in a tag. Here it is in Java, modified to have finite lookbehind (since Java doesn't support infinite length lookbehind).

    String s = "123 <o123o></o1o1> <oo 11='11x'> x11 <msg136='I <3 Johnny!11'>";
    System.out.println(
        s.replaceAll("(?<=<[^<>]{0,999})(?=[^<>]+>)1", "")
    ); // prints "123 <o23o></oo> <oo ='x'> x11 <msg136='I <3 Johnny!'>

There are many cases where this will fail, but it should get you started somewhere.

See also

Okay, I've "generalized" the problem so that it's not HTML related. Here's a snippet of Java that uses regex to remove [aeiou] from portions of a sentence enclosed by < and >, whose usage is reserved only to mark these special portions.

BEWARE: this regex is absolutely unreadable. But yes, it works. And it uses no lookbehind, too.

String s = "Wait <whaaat?> does this <really really> work???";
System.out.println(
    s.replaceAll("(?!>)(?:(?=<)|(?=\\G)(?!^))(?:(?:(?![aeiou])(.))|.)", "$1")
); // prints "Wait <wht?> does this <rlly rlly> work???"

I might try to explain it if there's interest, but otherwise I'd suggesting using a simple loop like this instead:

allocate output buffer
set isInside := false
for every character ch in input
   if (ch is openChar)
      isInside := true
   else if (ch is closeChar)
      isInside := false
   else if not (isInside and ch is control)
      append ch to buffer
polygenelubricants
  • 376,812
  • 128
  • 561
  • 623
  • Just out of interest, what in your opinion would the right tool be? I'm only using Regex because there's a pretty well defined pattern to what I'm trying to remove: i.e. "Remove all control characters from within angle brackets". I should also point out, my actual control character is not something that would appear in any valid html tag. – Iain Fraser May 12 '10 at 08:18
  • 1
    @Iain: if you can guarantee that there will never be `<` or `>` in attribute values, then regex is okay, I think. Otherwise you'll have trouble balancing quotes, handling escape sequences, etc, and I think it'll get dicey. If `<` and `>` is reserved for opening and closing tags only, then this is trivial. – polygenelubricants May 12 '10 at 08:40
  • Thanks for the advice polygenelubricants - your original regex worked perfectly by the way. My reluctance to use a parsing engine comes from the fact that you might not get valid starting and ending tags - they could be inconsistant. This is possible for example: `<1strong>Hey, I1'm in bold!` – Iain Fraser May 13 '10 at 00:26
1

You shouldn't generally use regex to parse html - but this is not html to begin with and hence you can't use a parser. The following seems to work.

var s = "The quick 1<strong>orange</strong> lemming <sp11a1n 1class1='jumpe111r'11>jumps over</span> 1the idle 1frog";
while(s.match(/<[^>]*?1(?=[^>]*>)/))
  s = s.replace(/(<[^>]*?)1(?=[^>]*>)/g, "$1");
console.log(s); //"The quick 1<strong>orange</strong> lemming <span class='jumper'>jumps over</span> 1the idle 1frog"
Amarghosh
  • 58,710
  • 11
  • 92
  • 121
  • Thanks for understanding Amarghosh - the idea is that I want to clean up a string so that it has a reasonable chance of parsing as html... – Iain Fraser May 12 '10 at 08:30
0

I get that you're not "parsing" it as such. You do however need to work out what is html tags and what isn't, this requires parsing and using a regex alone will not manage this.

Maybe the solution to the control chars in tag names is to replace globally all the control chars with a valid text pattern.

Then you can parse the resulting xml/html with an xml/html document parser. You can then run through this to perform your search and replaces on tagnames, attribute names, values.

Robin Day
  • 100,552
  • 23
  • 116
  • 167