0

I saw this question : How can I scrape pages with dynamic content using node.js?

I am trying to do the same with a little more complexity. There is a website which loads the dynamic content exactly like that but after a certain scroll, a stack comes which when clicked loads the dynamic content and I have to keep on clicking to get the result.

Here, is the link: http://www.flipkart.com/mobiles/pr?sid=tyy,4io&otracker=clp_mobiles_CategoryLinksModule_0-2_catergorylinks_11_ViewAll

Can someone help me how to achieve that? Thanks!

Community
  • 1
  • 1
Sumit Sahay
  • 504
  • 4
  • 22
  • 1
    You can try to port the suggestions in these two questions to phantomjs-node or a similar bridge between PhantomJS and node.js: [How to scroll down with Phantomjs to load dynamic content](http://stackoverflow.com/q/16561582/1816580) and [Scraping an infinite scroll page stops without scrolling](http://stackoverflow.com/q/29017726/1816580) – Artjom B. Apr 01 '16 at 09:47
  • 2
    You need to check the Ajx that they are hitting, and the json response. If you are lucky, u will get everything u need in the response json itself. – Parthapratim Neog Apr 01 '16 at 09:47
  • @ArtjomB. Thanks a lot, that indeed is a lot of help. Can it be done without phantom ie just using cheerio? – Sumit Sahay Apr 01 '16 at 11:14
  • No idea. I haven't used cheerio, but I don't think that it has a complete DOM+JavaScript runtime. I might be wrong. – Artjom B. Apr 01 '16 at 11:47

0 Answers0