-2

The title says it all. Please give me some way for how to scrape AJAX loaded Division.

There is one website which has Product Grid Division. I want this division for scrap the data from it but the problem is when i get the page content data in another html page there is no product grid because it's loaded via AJAX after some time. For Scrapping i tried both Perl and CURL with PHP.

Thanks in advance.

simbabque
  • 53,749
  • 8
  • 73
  • 136
Sahil Patel
  • 1,570
  • 2
  • 15
  • 34
  • What page (url) would you like to scrape? – PHPhil Jul 31 '15 at 10:44
  • PHPhil sorry i can't declare url – Sahil Patel Jul 31 '15 at 11:27
  • @SinanÜnür Thanks for advice but you don't know about me and you have to respect each and every programmer even if he/she fresher. I just want to know how to scrape the Division which is load with AJAX. I can't get that division id when i print that page content. I am scrapping data from last 4 months so i have enough skill to do this. If you don't know the answer then please don't comment. – Sahil Patel Jul 31 '15 at 12:03

2 Answers2

1

From WWW::Mechanize::FAQ

Which modules work like Mechanize and have JavaScript support?

In no particular order: Gtk2::WebKit::Mechanize, Win32::IE::Mechanize, WWW::Mechanize::Firefox, WWW::Scripter, WWW::Selenium

Also see: How do you scrape AJAX pages?

Community
  • 1
  • 1
Chankey Pathak
  • 21,187
  • 12
  • 85
  • 133
1

Using Selenium, e.g. through Selenium::Remote::Driver, you will be operating real browsers to access the site. Sites can be quite sensitive to subtle differences in browser behavior.

reinierpost
  • 8,425
  • 1
  • 38
  • 70