0

We are working on an application that is capable to screen scrape (crawl) all web pages for 10 sites.

We developed a tool using C# + Selenium + ChromeDriver and it is doing its job. The problem is that it is quite slow so we are looking for a way to make it faster.

Ideally, we would like to stop using Selenium and ChromeDriver.

The only reason why we used them was to actually render extracted HTML (Sometimes problem with plain HTML that we receive is that it has JavaScript that rebuilds HTML on page load).

So the question is how do we render extracted HTML in the most efficient way?

Stanislav
  • 165
  • 1
  • 15
  • If you are looking to interact with a JavaScript rendered HTML there is no better alternative then _Selenium_ and if you would like to stop using _Selenium_ you need to remove the `selenium` tag and start looking at `requests`. – undetected Selenium Mar 22 '19 at 20:39
  • @DebanjanB I don't actually think this is a duplicate of that question. – Stanislav Mar 22 '19 at 21:14
  • We can't think of a better alternative as neither you clarified your question nor you have responded to the counter question. – undetected Selenium Mar 22 '19 at 21:16
  • @Stanislav no need to delete your post. Duplicates are useful as a signpost to the original question. Your question seems to have different keywords, so it could be useful. Besides, that leaves you a chance to [edit] your question to explain why it isn't a duplicate, let others review it and possibly reopen – Jean-François Fabre Mar 22 '19 at 21:52
  • @Stanislav I am sure the dup target addresses all the points you have raised in the form of _problem is that it is quite slow_, _make it faster_, _JavaScript that rebuilds HTML on page load_, etc despite the fact your question lacking a [Minimal, Complete, and Verifiable example.](https://stackoverflow.com/help/mcve) – undetected Selenium Mar 22 '19 at 22:10

0 Answers0