Regex to extract info out of large html source?

Question

in among lots of html source i have some elements like this

<option value=15>Bahrain - Manama</option>
<option value=73>Bangladesh - Dhaka</option>
<option value=46>Barbados - Bridgetown</option>
<option value=285>Belarus - Minsk</option>
<option value=48>Belgium - Brussels</option>
<option value=36>Belize - Belmopan</option>

Also I have a dictionary declared like Dictionary<string, int> Places = new Dictionary<string, int>();

What I want to do it extract the City name out of the html and put it into of Places, and extract the number code out and put it into the int. For the first one I would add Placed.Add("Manama", 15); The country name can get ignored. The idea though is to scan the html source and add the Cities automatically.

this is what I have so far

string[] temp = htmlContent.Split('\n');
List<string> temp2 = new List<string>();
foreach (string s in temp)
{
    if (s.Contains("<option value="))
    {
        string t = s.Replace("option value=", ""); 
        temp2.Add(t); 
    }
}

This cuts out some of the text but then I more or less get stuck wondering how to extract the relevant parts from the text. It's really bad I know but I'm learning :(

You can try using [HTMLAgility Pack](http://htmlagilitypack.codeplex.com/) it will make it a little bit easier for you — Vamsi, Dec 11 '11 at 04:35
"Bahrain" is the *country* name. Do you want to ignore the country or the city? — Ben Voigt, Dec 11 '11 at 04:46
Kind of relevant (but always fun to read) [answer from another question](http://stackoverflow.com/a/1732454/80274). — Scott Chamberlain, Dec 11 '11 at 05:00

BrokenGlass · Answer 1 · 2011-12-11T04:55:56.247

Don't use a regular expression - use HtmlAgilityPack - now you can use Linq to retrieve your option elements and build up your dictionary in a one-liner:

HtmlDocument doc = new HtmlDocument();
//remove "option" special handling otherwise inner text won't be parsed correctly
HtmlNode.ElementsFlags.Remove("option"); 
doc.Load("test.html");

var Places = doc.DocumentNode
                .Descendants("option")
                .ToDictionary(x => x.InnerText.Split('-')[1].Trim(),
                              x => x.Attributes["value"].Value);

For extracting the city name from the option value the above uses string.Split(), splitting on the separating -, taking the second (city) string and trimming any leading or trailing whitespace.

score 0 · Accepted Answer · answered Dec 11 '11 at 04:53

If the only relevant data you are looking for is within

string[] options = Regex.Split(theSource, "<option value="); // Splits up the source which is downloaded from the url

that way you are instantly faced with an array of strings with the first few chars being your int. if the ints are always over 10, i.e 2 characters long, you can use:

int y = 2; // pointer
string theString = options[x].substring(0,2); // if the numbers are always > 10 its quicker than a loop otherwise leave this bit out and loop the is below
if(options[x].substring(y,1)!=">") // check to see if the number has finished
{
    theString += options[x].substring(y,1);
    y++;
}
int theInt = int.Parse(theString);

to get the number you can loop the if statement with a pointer if you need to get longer numbers. If the numbers are not always over 10, just loop the if statement with a pointer and ignore the first lines.

Then I would re-use the string theString:

string[] place = Regex.Split(options[x], " - "); // split it immediately after the name
theString = place[0].substring(y, place[0].length - y);

And then add them with

Places.Add(theString, theInt);

Shoud work, if the code doesnt work straigth away, the algorithms will, just make sure the spelling is right and that the variables are doing what they should

Regex to extract info out of large html source?

2 Answers2