How to retrieve data from a table on website?

Question

I've made a WinForms application which would get me a list of names from a table on a website. I am currently using a WebBrowser together with a Timer. And I think this could be done a lot smoother and faster. The WebBrowser is working slow (the old Internet Explorer one that is built-in), and it sometimes fail to get the data, and I have to run my timer again.

So I have a ListBox (which should contain the names). The ListBox is called PlayerList. Then I have a button, which activates the timer to grab the data. Here is my timer code.

private void UpdatePlayers_Tick(object sender, EventArgs e)
        {
            PlayerList.Items.Clear();
            if (this.Tibia.ReadyState == WebBrowserReadyState.Complete)
            {
                foreach (HtmlElement cell in this.Tibia.Document.GetElementsByTagName("tr"))
                {
                    string cls = cell.GetAttribute("className");
                    if (cls.StartsWith("Odd"))
                    {
                        dynamic oldname = cell.InnerText;
                        string[] strings = oldname.Split('0', '1', '2', '3', '4', '5', '6', '7', '8', '9');
                        string charnameonly = strings[0];
                        this.PlayerList.Items.Add(charnameonly);
                    }
                    else if (cls.StartsWith("Even"))
                    {
                        dynamic oldname = cell.InnerText;
                        string[] strings = oldname.Split('0', '1', '2', '3', '4', '5', '6', '7', '8', '9');
                        string charnameonly = strings[0];
                        this.PlayerList.Items.Add(charnameonly);
                    }
                }
            }
        }

I wonder if someone could help me achieve this, without a WebBrowser or something like that. Some code examples would be really nice.

Note: I only want the player names. Here is the website I get the data from: http://www.tibia.com/community/?subtopic=worlds&world=Antica

score 3 · Accepted Answer · answered Jan 07 '15 at 18:17

You can use HtmlAgilityPack

var players = await GetPlayers();

async Task<List<List<string>>> GetPlayers()
{
    string url = "http://www.tibia.com/community/?subtopic=worlds&world=Antica";
    using (var client = new HttpClient())
    {
        var html = await client.GetStringAsync(url);
        var doc = new HtmlAgilityPack.HtmlDocument();
        doc.LoadHtml(html);

        var table = doc.DocumentNode.SelectSingleNode("//table[@class='Table2']");
        return table.Descendants("tr")
                    .Skip(2)
                    .Select(tr => tr.Descendants("td")
                                    .Select(td => WebUtility.HtmlDecode(td.InnerText))
                                    .ToList())
                    .ToList();
    }
}

score 0 · Answer 2 · answered Jan 07 '15 at 18:22

0

Use Selenium. it's primarly designed for testing, and even better on scrapping data. Speaking from experience.

answered Jan 07 '15 at 18:22

Nadav Hury

564
4
12

How to retrieve data from a table on website?

2 Answers2