Running into an issue trying to extract links from htmlnode using htmlagiliypack

Question

This is a followup of a previous question I had.

I got the very excellent link parsing code from here.

So I have html of the following form:

<html>
    <head>
        RANDOM JAVASCRIPT AND CSS AHHHHHH!!!!!!!!
    </head>
    <body>
        <a href="/Random/link/here">Random</a>
        <a href="/Random/link/here">Random</a>
        <a href="/Random/link/here">Random</a>
        <a href="/Random/link/here">Random</a>
        <a href="/Random/link/here">Random</a>
        <a href="/Random/link/here">Random</a>
        <table class="table">
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
            <tr><a href="/subdir/members/Name">Name</a></tr>
        </table>
    <body>
</html>

And I have the following code, created with the purpose of extracting the information contained in , and then extracting the links fro that information:

public class MainClass
{
    public static void Main(String[] args)
    {
        string url = args[1];
        Extractinfo pageScrape = new Extractinfo();
        pageScrape.RenderPage(url);
    }
}
public class Extractinfo
{
    public HtmlDocument RenderPage(string url)
    {
        try
        {
                HtmlDocument pageSource = new HtmlDocument();
                var webGet = new HtmlWeb();
                pageSource = webGet.Load(url);

                ExtractLinks(pageSource);
        }
        catch (WebException e)
        {
            Console.WrtieLine(e.Message + ": " + e.StackTrace);
        }
    }

    private List<string> ExtractHrefTags(HtmlNode htmlSnippet)
        {
            List<string> hrefTags = new List<string>();

            foreach (HtmlNode link in htmlSnippet.SelectNodes("//a[@href]"))
            {
                HtmlAttribute att = link.Attributes["href"];
                hrefTags.Add(att.Value);
            }

            return hrefTags;
        }

        public void ExtractLinks(HtmlDocument pagesource)
        {

            var elements = pagesource.DocumentNode.SelectNodes("//table[@class='table']");
            List<string> hrefTags = new List<string>();
            foreach (var ele in elements)
            {
                 hrefTags = ExtractHrefTags(ele);
            }
        }
    }
}

Now, instead of getting only the links living inside of <table class="table>*****</table>, This code is putting all of the links on the page into List hreftags. What am I doing wrong here? How can I fix the error so that the only links extracted are the ones that live inside of <table class="table>*****</table>?

Thank you for your help!

score 1 · Accepted Answer · answered Jun 13 '12 at 16:29

1

You need to add a "." to your XPath to match the child nodes of the table, like this:

htmlSnippet.SelectNodes(".//a[@href]")

answered Jun 13 '12 at 16:29

shriek

5,157
2
36
42

I feel like a complete idiot... I was staring at this for like 30 minutes and couldn't figure it out. Thank you! – gfppaste Jun 13 '12 at 16:34

Running into an issue trying to extract links from htmlnode using htmlagiliypack

1 Answers1