I'm new to Python.
The aim of my script is to parse Web page with Selenium Webdriver. There are two subtasks. The first task I've made with no errors. But all script was without any function.
Now I'm trying to solve the second task in a little bit complicated way. I want to use functions.
The question is:
Could I pass an active Selenium Webdriver in a function parameter?
The second question is:
If the answer is YES - could the second function also pass an active Webdriver to another function?
So I think about activating a Webdriver and passing it into functions without closing a session.
My code is:
for key, value in enumerate(headers):
webdriver.DesiredCapabilities.PHANTOMJS[
'phantomjs.page.customHeaders.{}'.format(key)] = value
w = webdriver.PhantomJS()
ps = q_yn('Perform new links parsing?', default='no')
if ps:
csv_l = get_clist(w)
<...>
def get_clist(w):
w.get(url)
wait_pageload(w, 0)
src = w.page_source
BeautifulSoup(src, "lxml")
... (etc)
<..>
def wait_pageload(w, pt):
if pt == 0:
element = WebDriverWait(w, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "serg_search_result")))
elif pt == 1:
element = WebDriverWait(w, 10).until(
EC.presence_of_element_located((By.ID, "ya_share")))
else:
raise ValueError("Page type: %s is unknown" % pt)
When I try to build and run this script I see an error:
File "fts-crawl2.py", line 108, in wait_pageload
EC.presence_of_element_located((By.CLASS_NAME, "serg_s")))
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/support/
wait.py", line 71, in until
value = method(self._driver)
selenium.common.exceptions.WebDriverException: Message: Error - Unable
to load Atom 'find_element' from file ':/ghostdriver/./third_party/webdriver-
atoms/find_element.js'