Selenium: How to use Headless Chrome on AWS?

Question

Today I saw the message UserWarning: Selenium support for PhantomJS has been deprecated, please use headless versions of Chrome or Firefox instead.

I am willing to for for Chrome route. How do I install on AWS and run it on my remote AWS instance?

I will be using selenium in Python.

just add `chrome_options.add_argument("--headless")` or use docker image for chrome-standalone — Senior Pomidor, Jan 31 '18 at 08:17
See the discussion https://stackoverflow.com/questions/46085270/selenium-common-exceptions-webdriverexception-message-chromedriver-executabl/46089751#46089751 — undetected Selenium, Jan 31 '18 at 08:18
Well, your question seems unclear. 1) Heading speaks about _Use Headless Chrome on AWS_. 2) Body speaks about _install on AWS_ 3) Tag on _selenium_ . What is your exact question? — undetected Selenium, Jan 31 '18 at 12:03

score 15 · Answer 1 · answered Apr 13 '18 at 05:20

15

Create a new EC2 instance.

SSH log into the machine.

Install python, selenium, chromedriver, chromium, and python packages what you need.

sudo apt install chromium-chromedriver

Copy your python script to the machine.

Edit the script and add an chromeoption.

import selenium as se

options = se.webdriver.ChromeOptions()
options.add_argument('headless')

driver = se.webdriver.Chrome(chrome_options=options)

Done!

answered Apr 13 '18 at 05:20

Leon

634
1
7
22

I see this when I do that: `selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH.` – Ed Summers Sep 21 '18 at 20:19
1

@EdSummers Find chromedriver executable path and add the path into your PATH environment variable. – Leon Nov 28 '18 at 06:11
@Leon how to do that? – greendino Apr 22 '22 at 02:40
Here is a good article about path: [How to set your $PATH variable in Linux](https://opensource.com/article/17/6/set-path-linux) – Leon Apr 23 '22 at 06:05

score 4 · Answer 2 · answered Nov 21 '19 at 23:46

4

I prefer to use Firefox so this is my Python3 implementation

def createHeadlessFirefoxBrowser():
     options = webdriver.FirefoxOptions()
     options.add_argument('--headless')
     return webdriver.Firefox(options=options)

browser = createHeadlessFirefoxBrowser()

answered Nov 21 '19 at 23:46

Peter

202
2
5

how did you get the driver? – greendino Apr 22 '22 at 02:48

Selenium: How to use Headless Chrome on AWS?

2 Answers2

Linked