c# - reading HTML?

Question

I'm developing a program in C# and I require some help. I'm trying to create an array or a list of items, that display on a certain website. What I'm trying to do is read the anchor text and it's href. So for example, this is the HTML:

<div class="menu-1">
    <div class="items">
        <div class="minor">
            <ul>
                <li class="menu-item">
                    <a class="menu-link" title="Item-1" id="menu-item-1"
                    href="/?item=1">Item 1</a>
                </li>
                <li class="menu-item">
                    <a class="menu-link" title="Item-1" id="menu-item-2"
                    href="/?item=2">Item 2</a>
                </li>
                <li class="menu-item">
                    <a class="menu-link" title="Item-1" id="menu-item-3"
                    href="/?item=3">Item 3</a>
                </li>
                <li class="menu-item">
                    <a class="menu-link" title="Item-1" id="menu-item-4"
                    href="/?item=4">Item 4</a>
                </li>
                <li class="menu-item">
                    <a class="menu-link" title="Item-1" id="menu-item-5"
                    href="/?item=5">Item 5</a>
                </li>
            </ul>
        </div>
    </div>
</div>

So from that HTML I would like to read this:

string[,] array = {{"Item 1", "/?item=1"}, {"Item 2", "/?item=2"},
    {"Item 3", "/?item=3"}, {"Item 4", "/?item=4"}, {"Item 5", "/?item=5"}};

The HTML is an example I had written, the actual site does not look like that.

Did you try to take a look to the XmlTextReader stream? You'll catch all the **a** and - plus - it's quick even with a big XML file. — Adriano Repetti, May 22 '12 at 20:11

score 9 · Accepted Answer · answered May 22 '12 at 20:35

9

As others said HtmlAgilityPack is the best for html parsing, also be sure to download HAP Explorer from HtmlAgilityPack site, use it to test your selects, anyway this SelectNode command will get all anchors that have ID and it start with menu-item :

  HtmlDocument doc = new HtmlDocument();
  doc.Load(htmlFile);
  var myNodes = doc.DocumentNode.SelectNodes("//a[starts-with(@id,'menu-item-')]");
  foreach (HtmlNode node in myNodes)
  {
    Console.WriteLine(node.Id);

  }

answered May 22 '12 at 20:35

Antonio Bakula

20,445
6
75
102

The HAP Explorer ? Where do you get that from ? – paul1923 Sep 03 '21 at 07:15
try here https://github.com/tomap/HtmlAgilityPack/tree/master/HAPExplorer – Antonio Bakula Sep 06 '21 at 18:26

score 2 · Answer 2 · answered May 22 '12 at 20:18

2

If the HTML is valid XML you can load it using the XmlDocument class and then access the pieces you want using XPaths, or you can use and XmlReader as Adriano suggests (a bit more work).

If the HTML is not valid XML I'd suggest to use some existing HTML parsers - see for example this - that worked OK for us.

answered May 22 '12 at 20:18

MiMo

11,793
1
33
48

[XML Validator](https://www.w3schools.com/xml/xml_validator.asp) – Toni Apr 28 '21 at 15:50

score 1 · Answer 3 · answered May 22 '12 at 20:24

1

You can also use the HtmlAgility pack

answered May 22 '12 at 20:24

Gregoire

24,219
6
46
73

score 1 · Answer 4 · answered May 22 '12 at 20:33

1

I think this case is simple enough to use a regular expression, like <a.*title="([^"]*)".*href="([^"]*)":

string strRegex = @"<a.*title=""([^""]*)"".*href=""([^""]*)""";
RegexOptions myRegexOptions = RegexOptions.None;
Regex myRegex = new Regex(strRegex, myRegexOptions);

string strTargetString = ...;

foreach (Match myMatch in myRegex.Matches(strTargetString))
{
  if (myMatch.Success)
  {
    // Use the groups matched
  }
}

answered May 22 '12 at 20:33

Helstein

330
2
6

http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags – MiMo May 22 '12 at 22:22

c# - reading HTML?

4 Answers4

Linked