Here, I am trying to select div.pager with jsoup but it is not showing up and gives no value. My code:
Element mainDiv = doc.select("div.mangaread-main > div.pager").first();
Asked
Active
Viewed 20 times
0
Shivansh Potdar
- 1,146
- 6
- 16
-
Print `doc` to console and see if you can find that element there. Some sites add elements dynamically via JavaScript code, but Jsoup is not browser emulator and can't execute that code. In that case you would need other tools like Selenium webdriver. – Pshemo Aug 12 '20 at 15:56
-
Yea that seems to be the matter here, thanks @Pshemo – Shivansh Potdar Aug 12 '20 at 16:00
-
If you're technically capable, or want to provide the URL in use I can take a look... Use Chrome Dev Tools and view the requests going out via the Network tab.. you'll find whether you're looking at the right URL by looking at what requests are made, what the responses are, etc. Typically there's some dynamic document manipulation going on if you don't get the result you expect with JSoup. You then need to do some investigation to find the correct request URL. HtmlUnit I find works better in some scenarios where Javascript is present. – Rob Evans Aug 13 '20 at 10:52
-
@RobEvans Thanks for the suggestion I will look into using HtmlUnit – Shivansh Potdar Aug 16 '20 at 15:11