20

my code is: 

from selenium import webdriver

driver = webdriver.PhantomJS(executable_path='driver/bin/phantomjs.exe')
driver.get("https://www.test.com")
print(driver.current_url)

It seems to run fine but before it runs I always get this error:

UserWarning: Selenium support for PhantomJS has been deprecated, please use headless versions of Chrome or Firefox instead warnings.warn('Selenium support for PhantomJS has been deprecated, please use headless

Why am I getting this error? I thought my PhantomJS was headless as it still works and no browser pops-up is this error save to ignore?

Ronak Shah
  • 377,200
  • 20
  • 156
  • 213
Ogden
  • 477
  • 1
  • 8
  • 21
  • PhantomJS is based on an old version of webkit and is no longer being maintained. https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE – generalhenry May 18 '18 at 17:35
  • Ah ok, @generalhenry is there an easy way to make Chrome or Firefox headless? – Ogden May 18 '18 at 17:37
  • https://intoli.com/blog/running-selenium-with-headless-chrome/ short version install chrome, chromedriver then in your code call `driver = webdriver.Chrome . . .` and the rest of your code should function as is. – generalhenry May 18 '18 at 17:41
  • or https://stackoverflow.com/questions/46753393/how-to-make-firefox-headless-programatically-in-selenium-with-python for firefox – generalhenry May 18 '18 at 17:42
  • but still PhontomJS is much faster and needed for Large scale carawling project. – softmarshmallow Aug 22 '18 at 06:02

4 Answers4

26

Selenium considers PhantomJS as deprecated, so you need to us either Chrome or Firefox in headless mode.

Here are the steps to use Chrome in headless mode:

  1. download chrome driver from https://sites.google.com/a/chromium.org/chromedriver/getting-started
  2. extract it to a folder
  3. add this folder to your PATH environment variable (if you don't do it, you will have to use webdriver.Chrome('/your/path/to/chromedriver') in the code below instead of webdriver.Chrome())

Then use it like this:

from selenium import webdriver

# prepare the option for the chrome driver
options = webdriver.ChromeOptions()
options.add_argument('headless')

# start chrome browser
browser = webdriver.Chrome(chrome_options=options)
browser.get('http://www.google.com/xhtml')
print(browser.current_url)
browser.quit()

More on how to use ChromeDriver
For the other options: here (also here and here)

MagTun
  • 5,619
  • 5
  • 63
  • 104
  • This method now also seems to be deprecated. I tried it, and it said this: ```DeprecationWarning: use options instead of chrome_options browser = webdriver.Chrome(chrome_options=options)``` – Ryan Hart Jul 18 '20 at 00:24
  • @RyanHart, what is your version of ChromeDriver and selenium? I have `ChromeDriver 84.0.4147.30` and selenium `3.141.0`(I updated both of them just now). I just checked the code in my answer and I don't have any error. If you still have an error try `browser = webdriver.Chrome(options=options)` instead of `browser = webdriver.Chrome(chrome_options=options)` and let me know if you still get an error – MagTun Jul 18 '20 at 12:03
  • I have those exact versions installed as well, but it still gives me that deprecation warning. I did, however, get it to work by removing the `options` lines as well as all parameters in `webdriver.Chrome()`. So, I don't need any more help. Perhaps you have warnings like these suppressed in your settings somehow. I don't know. Everything is good on my end though, now. – Ryan Hart Jul 18 '20 at 21:18
  • 1
    Oh you're right. `options=options` fixed the deprecation warning message. When I tried earlier, I hadn't even scripted anything yet, so all I saw was the warning message an assumed it wouldn't work. I tried tinkering with it again since I decided I wanted to disable all images and javascript to speed up my script. – Ryan Hart Jul 20 '20 at 07:26
1

In Selenium 3.8.1 PhantomJS marked as deprecated webdriver and recommend us to use either Chrome or Firefox in headless mode.

Sergey Samsonov
  • 19
  • 2
0

You could use this:

from selenium import webdriver

browser = webdriver.Chrome('chromedriver_path/chromedriver')
browser.get("https://www.test.com")
print(browser.current_url)
browser.quit()
jarh1992
  • 603
  • 7
  • 8
-1

Found an alternative you can add options.add_argument('headless') to chrome

Samsul Islam
  • 2,581
  • 2
  • 17
  • 23
Ogden
  • 477
  • 1
  • 8
  • 21