2

I am doing some testing on my site, and I have a python program which does gets on few different pages. Some of these pages have $(document).ready(function(). I noticed that when I do get through python, I get the code, but for example $(document).ready(function() doesn't run.

How can I run the $(document).ready(function() of the site I am doing a GET on?

Thank you for help.

Amit Singh
  • 2,267
  • 4
  • 25
  • 50
emraldinho
  • 475
  • 8
  • 23
  • what are you using to do GET? see this: http://stackoverflow.com/questions/20622870/using-urllib2-to-execute-url-and-return-rendered-html-output-not-the-html-itsel – gabereal Nov 23 '14 at 06:20
  • if you can use node.js instead of python you could use PhantomJS which is a headless browser. – gabereal Nov 23 '14 at 06:21
  • 1
    Related: [Headless Browser for Python](http://stackoverflow.com/questions/6025082/headless-browser-for-python-javascript-support-required), [Screen Scraping a Javascript based webpage in Python](http://stackoverflow.com/questions/8183682/screen-scraping-a-javascript-based-webpage-in-python), [Screen scraping with Python](http://stackoverflow.com/questions/2190502/screen-scraping-with-python) – Jonathan Lonowski Nov 23 '14 at 06:25
  • headless browser! that's what I should have searched for, thank you very much – emraldinho Nov 23 '14 at 06:30

1 Answers1

1

You should go for Selenium, it lets you control a real browser from your python code . That means your javascript will be executed by the browser .

Example code :

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.send_keys("pycon")
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
driver.close()
Alexander
  • 12,424
  • 5
  • 59
  • 76