0

The onclick thing

Hi guys, I'm stuck when I'm trying to scrape a webpage. There are reviews, and some have a button "more". I'd love to scrape the whole review. Here shows the html code. I got an onclick with a function for the button "more".

I'm using python and requests. Anyone can help me about how to do it?

PS: How to differentiate between jQuery and Ajax? I think here is jQuery right?

Thanks very much for your help!

peterh
  • 11,875
  • 18
  • 85
  • 108
Elona Mishmika
  • 480
  • 2
  • 5
  • 21
  • 1
    I guess you have to trigger the onclick event for each "button" and get the data it brings... this question may give you a heads up: [invoke javascript function when scraping the web page](http://stackoverflow.com/questions/8845423/invoke-javascript-function-when-scraping-the-web-page) – DIEGO CARRASCAL Aug 18 '16 at 15:20
  • Thanks, but could you tell me a little about the mechanism? Seems that I need to use Sellenium package. But is it because I'll need to send a request to the server? – Elona Mishmika Aug 19 '16 at 12:18
  • 1
    The first time the Web gets loaded, does it comes with the text you want? I assume it doesn't, so it must somehow request that information... the easy way will be to trigger the functions being called by each button, so you create an array (or query) of the buttons, call the event and read again... this is how you call the event in javascript: [How can I trigger a JavaScript event click](http://stackoverflow.com/questions/2381572/how-can-i-trigger-a-javascript-event-click). Well is an idea, there may be better ways to doit from Python... – DIEGO CARRASCAL Aug 19 '16 at 14:32
  • 1
    The easiest way would probably be to control a web browser from Python using something like selenium (which has Python bindings) or write your scraper in nodejs and use some PhantomJS based client like zombie. The requests module only knows HTTP, it can’t run a DOM for you. – binki Aug 20 '16 at 23:37

0 Answers0