Faster way to parse JS generated site

Asked Jan 29 '14 at 16:35

Active Jan 29 '14 at 20:56

Viewed 81 times

I am creating a script that will have to parse from a JS generated site ( using python ) atleast a 1000 times a day. Parsing it the usual way (using a browser to open it and then get the code) takes me about 30 seconds. Which is not satisfactory. I thought about how to make the process faster and I have an idea - is it possible to create a browser that doesn`t create a window ( ignores the visual part ) and is simply a process in other words an "invisible" browser. What I want to know is if it will be efficient and if there are other ways to make it run faster. Any help is appreciated.

EDIT : Here is the code for my parser

from selenium import webdriver
import re
browser = webdriver.Firefox()
browser.get('http://www.spokeo.com/search?q=Joe+Henderson,+Phoenix,+AZ&sao7=t104#:18643819031')
content = browser.page_source
browser.quit()

edited Jan 29 '14 at 20:56

asked Jan 29 '14 at 16:35

Peter Lazarov

Please check this question: http://stackoverflow.com/questions/390992/javascript-parser-in-python – avenet Jan 29 '14 at 18:41
I have a parser already I just need to make it faster – Peter Lazarov Jan 29 '14 at 18:47
Are you using a library to implement/generate your parser or did you implemented one yourserlf? – avenet Jan 29 '14 at 18:49
I just posted the code for my parser – Peter Lazarov Jan 29 '14 at 20:56
Sorry Peter, but I don't understand what you need to parse the Javascript: to execute it later or to check if it's correct. – avenet Jan 29 '14 at 21:00
It works correctly. What i am looking for is a way to make it faster. – Peter Lazarov Jan 29 '14 at 21:44
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/46427/discussion-between-avenet-and-peter-lazarov) – avenet Jan 30 '14 at 12:11

Faster way to parse JS generated site

0 Answers0