2

I am new to programming and am trying to scrape data off a web-page which appears to be loaded by page executed javascript.

I am told selenium is a good bet for being able to do this

I use Jupyter with Anaconda

I have installed selenium by using conda install

However, when trying to use the webdriver, I am running into a problem

from selenium import webdriver

driver = webdriver.firefox()

The error that shows is:

WebDriverException: Message: 'geckodriver' executable needs to be in PATH.

What can I do about this?

Thanks

dpapadopoulos
  • 1,834
  • 5
  • 23
  • 34
Michael
  • 343
  • 2
  • 7

1 Answers1

0

You need to download geckodriver from here (Firefox browser and geckodriver need to integrate properly, so download the proper versions for both) and do the following:

# Generic way
driver = webdriver.Firefox(PATH_TO_YOUR_DOWNLOADED_GECKODRIVER)

# Windows example below
driver = webdriver.Firefox("C:\\Users\\YOUR_USERS_ACCOUNT_NAME_IN_WINDOWS\\ANY_DIRECTORY_YOU_WANT_UNDER_THIS_ACCOUNT\\geckodriver.exe")

# Linux example below
driver = webdriver.Firefox("usr\\local\\bin\\geckodriver")

Also, you need capital F for Firefox and replace "\" with double "\\" as it is in the examples above.

Edit

Tip: download geckodriver inside a directory with the proper permissions. Then unzip the folder, locate the exe file and its path. Then insert this path into the parenthesis. Please use the double and not the single lines as I told you above.

halfer
  • 19,824
  • 17
  • 99
  • 186
dpapadopoulos
  • 1,834
  • 5
  • 23
  • 34
  • 1
    Thanks. I have downloaded and installed into the folder you used in your windows example. I am now running into error message: NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\Public\\geckodriver.exe'. What can I do about this? If I replace with a single backstroke then I get a unicode error – Michael Feb 23 '19 at 01:27