0

I am scraping web sites with HtmlAgilityPack but some websites run javascript after page load. So HtmlAgility does not help at this point. I have tried selenium web driver and NHtmlUnit and PhantomJS. But they are too slow for my task.

I also tried web browser but for asp.net it is slow and not a good solution for me. Also it does work for a few days after that that site has gived an error: "Javascript must be enabled"

So what do you suggest for me to do scrape those web sites fast enough?

PeerNet
  • 855
  • 1
  • 16
  • 31
  • Possible duplicate of [Running Scripts in HtmlAgilityPack](https://stackoverflow.com/questions/11393075/running-scripts-in-htmlagilitypack) – mjwills Jul 06 '17 at 09:19
  • @mjwills I edited my answer. I tried web browser but I am coding asp.net webform. So it is a little bit weird for me. Also it works for me for a few days but after those web sites give an error "Javascript must be enabled" – PeerNet Jul 06 '17 at 09:23

0 Answers0