Parsing out content from HTML using regex?

Question

How can I use regex to find everything except for data within div with a specific style? e.g.

<div style="float:left;padding-left:10px; padding-right:10px">
    <img src="../Style/BreadCrumbs/Divider.png">
</div>
<div style="float:left; padding-top:5px;">
    Data to keep
</div>
<div style="float:left;padding-left:10px; padding-right:10px">
    <img src="../Style/BreadCrumbs/Divider.png">
</div>

I want regex to match everything except for the data. The best way I can see is to just remove the html markup and combine the files afterwards with vb (I already have the code for vb.)

I'm using regex because I need to extract the data from several hundred files.

http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454 — Mat, Feb 28 '12 at 11:18
Provide some html to parse. And ignore all the crap you're going to hear about not parsing html with regex, it's perfectly applicable in simple cases ;) — demorge, Feb 28 '12 at 11:20
If you can make strong assumptions about the content of the div, you can extract it, but you need to specify those assumptions here, along with an example. — beerbajay, Feb 28 '12 at 12:03

score 1 · Accepted Answer · answered Feb 28 '12 at 15:44

Your suggested method is probably not a good way to do this. If:

you have access to grep
your version of grep supports perl-compatible regex (PCRE)
this style of div only wraps your data, not other elements
the 'data' div does not contain other divs

Then you can use:

(?s)<div style="float:left; padding-top:5px;">.*?</div>

The important parts of this are:

(?s) which activates DOTALL, which means that . will match newlines
.*? which matches the contents of the div reluctantly, which means it'll stop at the first </div> it finds.

To use this, you'll need to activate a few grep options:

grep -Pzo $PATTERN file

For these:

-P activates the PCRE
-z replaces \n by NUL so grep will treat the entire file as a single line
-o prints only the matching parts

After this you'll need to strip off the divs. sed is a good tool for this.

sed 's|</\?div[^>]*>||g'

If you put all of your files in one directory you can do the joining at the same time:

grep -Pzo $PATTERN *.html | sed 's|</\?div[^>]*>||g' > out.html

@sas Welcome to SO! A common way to say thanks here is by up-voting the answer, and accepting by clicking the check-mark (if it's the best answer.) — Rob, Feb 28 '12 at 20:28

Parsing out content from HTML using regex?

1 Answers1

Linked