0

I need to write a c++ program that gets data from a web page (i have the url to that). Basically the web page contains a huge table spread over multiple pages (though next buttons).

I need to get this entire table and then take each row and parse it (table has around 10 columns). Each of this has to be put into a csv file in a similar data format after some minor processing.

Can anybody suggest the best way to do this. I need to do this in c++ am wondering what libs i shd use. Needs to be done in windows but i dont mind using platform independent / dependent stuff.

Thanks in advance.

AMM
  • 17,130
  • 24
  • 65
  • 77

2 Answers2

2

Let libcurl do the downloading for you. It's fast and you don't have to deal with HTTP directly.

There are many libraries out there for parsing XML in C++. See this.

Do everybody a favour and don't try to parse HTML with a regexp. You will summon Cthulhu.

Community
  • 1
  • 1
pmr
  • 58,701
  • 10
  • 113
  • 156
  • I would like to know how to do the page navigation. The table is spread across multiple pages and there is a bunch of next, page numbers at the bottom of the table. Now these map to some javascript coee that calls a function called __doPostBack(some args) which in-turn will call the form submit. I want to know how to achieve these. – AMM Apr 19 '11 at 18:44
0

I know that this might not be the exact answer you are looking for, but since it is an option...

You could generate a basic HTTP GET with a raw socket and then you would receive your table in HTML format. You would then only have to parse it (using an XML parser or manualy looking for the table or a specific ID) and get your results.

I know this is not the best and "ready to use" solution, but still it could work.

Drahakar
  • 5,986
  • 6
  • 43
  • 58