Programmable WebCrawler with C#

Question

I would like to extract specific data form a known Url : from html tags like span, a, divs ... !
So I'm searching for an existing library for a WebCrawler written in C# ! ... or maybe any other idea.

Thank you !

score 2 · Accepted Answer · answered Jun 27 '11 at 16:38

2

For parsing HTML, you might want to look into the HTML Agility Pack. As for grabbing the HTML from a website, you can use WebClient.

answered Jun 27 '11 at 16:38

NickAldwin

11,584
12
52
67

I would like to parse HTML with C# just like we do with DOM ! Like for every Div Having the specific class name I would like to get text form child span or a or any other tags ! – Aladdin Gallas Jun 27 '11 at 17:28
@Aladdin Gallas I'm not sure what you mean...HTML Agility Pack ****can**** parse HTML. – NickAldwin Jun 27 '11 at 17:29
@NickAldwin : If HTMLAgilityPack can do that, I just need an example ! Thank you so much for Helping me – Aladdin Gallas Jun 27 '11 at 17:33
@Aladdin Gallas http://stackoverflow.com/questions/846994/how-to-use-html-agility-pack – NickAldwin Jun 27 '11 at 18:33
@Aladdin Gallas if you have a specific question about using Html Agility Pack, ask it separately on SO. – NickAldwin Jun 27 '11 at 18:33
Thank you so much NickAldwin. I have to test it before and if I have issues I'll told you. – Aladdin Gallas Jun 27 '11 at 21:45

score 0 · Answer 2 · answered Jun 27 '11 at 16:37

0

This guy wrote an article on using HtmlAgilityPack for web crawling from C#, and this is a framework for web crawling, though the project seems to be dead, you can look at them and see if they fit your needs.

answered Jun 27 '11 at 16:37

james_bond

6,778
3
28
34

score 0 · Answer 3 · answered Jun 27 '11 at 16:37

0

A lot of people on SO like HtmlAgilityPack.

answered Jun 27 '11 at 16:37

Chris Trombley

2,232
1
17
24

Programmable WebCrawler with C#

3 Answers3