How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?

Question

I'm working on a python script to web-scrape and have gone down the path of using Chromedriver as one of the packages. I would like this to operate in the background without any pop-up windows. I'm using the option 'headless' on chromedriver and it seems to do the job in terms of not showing the browser window, however, I still see the .exe file running. See the screenshot of what I'm talking about. Screenshot

This is the code I am using to initiate ChromeDriver:

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches",["ignore-certificate-errors"])
options.add_argument('headless')
options.add_argument('window-size=0x0')
chrome_driver_path = "C:\Python27\Scripts\chromedriver.exe"

Things I've tried to do is alter the window size in the options to 0x0 but I'm not sure that did anything as the .exe file still popped up.

Any ideas of how I can do this?

I am using Python 2.7 FYI

Possible duplicate of [Running Selenium with Headless Chrome Webdriver](https://stackoverflow.com/questions/53657215/running-selenium-with-headless-chrome-webdriver) — Basj, Oct 14 '19 at 08:29
@Basj This question is an year older than the question you linked. If anything, the linked question would be a possible duplicate of this. — Allan Lago, Dec 03 '19 at 14:33
I know its a bad idea to like reply to question which is 4 years old. but i see nobody actually solving issue of being shown. If the platform is windows you can do: `import win32gui` and `import win32.lib.win32con as win32con` and in the code include something like `Hwnd = win32gui.FindWindowEx(None,None,None,chrome_driver_path)` and then `win32gui.ShowWindow(Hwnd,win32con.SW_HIDE)` later if you want to show it again, you need to `win32gui.ShowWindow(Hwnd,win32con.SW_SHOW)` The code will completely hide the window. only viewable through programs such as task manager running in background — Pear, Nov 09 '21 at 07:28

Daniel Porteous · Answer 1 · 2023-05-23T15:17:05.457

It should look like this:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')  # Last I checked this was necessary.
driver = webdriver.Chrome(CHROMEDRIVER_PATH, chrome_options=options)

This works for me using Python 3.6, I'm sure it'll work for 2.7 too.

Update 2018-10-26: These days you can just do this:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)

Update 2023-05-22: Chrome’s Headless mode got an upgrade and now both headless and headful modes have been unified under the same implementation. See https://developer.chrome.com/articles/new-headless/

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--headless=new')
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)

To try the new Headless mode, pass the --headless=new command-line flag:

chrome --headless=new

For now, the old Headless mode is still available via:

chrome --headless=old

Currently, passing the --headless command line flag without an explicit value still activates the old Headless mode — but we plan to change this default to new Headless over time.

Thanks! This unfortunately didn't solve the issue. I have posted my answer to what did though. Appreciate your help — Maz, Oct 27 '17 at 00:53
Latest update works when replacing: options = Options() with options = webdriver.ChromeOptions() — TomKivy, Apr 03 '19 at 21:35
Update: kwarg `chrome_options` for Chrome is deprecated in favour of `options` — gavin, Mar 24 '21 at 08:06
The solution does not work when the website has javascript. I added `options.add_argument('user-agent=fake-useragent')` and it worked. — jaycode, Mar 22 '22 at 14:40

undetected Selenium · Answer 2 · 2021-10-16T14:31:30.387

Answer update of 13-October-2018

To initiate a google-chrome-headless browsing context using Selenium driven ChromeDriver now you can just set the --headless property to true through an instance of Options() class as follows:

Effective code block:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True
driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
driver.get("http://google.com/")
print ("Headless Chrome Initialized")
driver.quit()

Answer update of 23-April-2018

Invoking google-chrome in headless mode programmatically have become much easier with the availability of the method set_headless(headless=True) as follows :

Documentation :

set_headless(headless=True)
    Sets the headless argument

    Args:
        headless: boolean value indicating to set the headless option

Sample Code :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.set_headless(headless=True)
driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
driver.get("http://google.com/")
print ("Headless Chrome Initialized")
driver.quit()

Note : --disable-gpu argument is implemented internally.

Original Answer of Mar 30 '2018

While working with Selenium Client 3.11.x, ChromeDriver v2.38 and Google Chrome v65.0.3325.181 in Headless mode you have to consider the following points :

You need to add the argument --headless to invoke Chrome in headless mode.
For Windows OS systems you need to add the argument --disable-gpu
As per Headless: make --disable-gpu flag unnecessary --disable-gpu flag is not required on Linux Systems and MacOS.
As per SwiftShader fails an assert on Windows in headless mode --disable-gpu flag will become unnecessary on Windows Systems too.
Argument start-maximized is required for a maximized Viewport.
Here is the link to details about Viewport.
You may require to add the argument --no-sandbox to bypass the OS security model.

Effective windows code block :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--headless") # Runs Chrome in headless mode.
options.add_argument('--no-sandbox') # Bypass OS security model
options.add_argument('--disable-gpu')  # applicable to windows os only
options.add_argument('start-maximized') # 
options.add_argument('disable-infobars')
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\path\to\chromedriver.exe')
driver.get("http://google.com/")
print ("Headless Chrome Initialized on Windows OS")

Effective linux code block :

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--headless") # Runs Chrome in headless mode.
options.add_argument('--no-sandbox') # # Bypass OS security model
options.add_argument('start-maximized')
options.add_argument('disable-infobars')
options.add_argument("--disable-extensions")
driver = webdriver.Chrome(chrome_options=options, executable_path='/path/to/chromedriver')
driver.get("http://google.com/")
print ("Headless Chrome Initialized on Linux OS")

Steps through YouTube Video

How to initialize Chrome Browser in Maximized Mode through Selenium

Outro

How to make firefox headless programmatically in Selenium with python?

tl; dr

Here is the link to the Sandbox story.

One more edit needed here use `chrome_options=options` not `options=options` — Jortega, Jan 08 '20 at 17:47
Another one: use `options.add_argument("--headless")` not `options.headless = True` — Jortega, Jan 08 '20 at 17:53
@DebanjanB, When I use your 13-Oct-18 code it executes, but throws the following errors: ` "Error parsing a meta element's content: ';' is not a valid key-value pair separator. Please use ',' instead."` and `"Scripts may close only the windows that were opened by it."` and `"Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience...` for the website `https://test.plexonline.com` - browser with a 'head' executes with no errors though... any ideas? — Programming_Learner_DK, Feb 04 '20 at 12:32
@Python_Learner_DK The compete error stack trace may give us some clue. Can you raise a new question along with your binary versions please? — undetected Selenium, Feb 04 '20 at 12:45
@DebanjanB, here is the new question: https://stackoverflow.com/questions/60058246/going-headless-in-selenium-with-chrome-using-python-executing-but-throwing-erro — Programming_Learner_DK, Feb 04 '20 at 13:15
@Jortega, chrome_options is depreciated so options=options is correct. — Rene, Apr 03 '20 at 00:49

Eduardo Luis Santos · Answer 3 · 2023-07-11T14:34:08.353

34

Update August 20, 2020 (still valid) -- Now is simple!

chrome_options = webdriver.ChromeOptions()
chrome_options.headless = True

self.driver = webdriver.Chrome(
            executable_path=DRIVER_PATH, chrome_options=chrome_options)

edited Jul 11 '23 at 14:34

answered Aug 10 '20 at 23:27

Eduardo Luis Santos

506
4
9

score 16 · Answer 4 · answered Dec 09 '20 at 09:16

16

UPDATED It works fine in my case:

from selenium import webdriver

options = webdriver.ChromeOptions()
options.headless = True
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)

Just changed in 2020. Works fine for me.

answered Dec 09 '20 at 09:16

Zubayer

571
1
8
16

score 10 · Accepted Answer · answered Oct 27 '17 at 00:51

So after correcting my code to:

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches",["ignore-certificate-errors"])
options.add_argument('--disable-gpu')
options.add_argument('--headless')
chrome_driver_path = "C:\Python27\Scripts\chromedriver.exe"

The .exe file still came up when running the script. Although this did get rid of some extra output telling me "Failed to launch GPU process".

What ended up working is running my Python script using a .bat file

So basically,

Save python script if a folder
Open text editor, and dump the following code (edit to your script of course)

c:\python27\python.exe c:\SampleFolder\ThisIsMyScript.py %*
Save the .txt file and change the extension to .bat
Double click this to run the file

So this just opened the script in Command Prompt and ChromeDriver seems to be operating within this window without popping out to the front of my screen and thus solving the problem.

score 6 · Answer 6 · answered Oct 25 '17 at 10:11

The .exe would be running anyway. According to Google - "Run in headless mode, i.e., without a UI or display server dependencies."
Better prepend 2 dashes to command line arguments, i.e. options.add_argument('--headless')
In headless mode, it is also suggested to disable the GPU, i.e. options.add_argument('--disable-gpu')

score 4 · Answer 7 · answered May 08 '20 at 11:14

Try using ChromeDriverManager

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager 
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.set_headless()
browser =webdriver.Chrome(ChromeDriverManager().install(),chrome_options=chrome_options)
browser.get('https://google.com')
# capture the screen
browser.get_screenshot_as_file("capture.png")

score 3 · Answer 8 · edited Nov 20 '20 at 07:41

3

Solutions above don't work with websites with cloudflare protection, example: https://paxful.com/fr/buy-bitcoin.

Modify agent as follows: options.add_argument("user-agent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36")

Fix found here: What is the difference in accessing Cloudflare website using ChromeDriver/Chrome in normal/headless mode through Selenium Python

edited Nov 20 '20 at 07:41

Ryan M

18,333
31
67
74

answered Nov 05 '20 at 11:14

Elho

31
2

I was working with pexels.com and came up with exactly this scenario. This works like a charm. – Syed Muntazir Mehdi Jan 31 '23 at 08:54

Red Gundu · Answer 9 · 2020-07-30T01:47:13.450

  
from chromedriver_py import binary_path
 
 
chrome_options = webdriver.ChromeOptions()
   chrome_options.add_argument('--headless')
   chrome_options.add_argument('--no-sandbox')
   chrome_options.add_argument('--disable-gpu')
   chrome_options.add_argument('--window-size=1280x1696')
   chrome_options.add_argument('--user-data-dir=/tmp/user-data')
   chrome_options.add_argument('--hide-scrollbars')
   chrome_options.add_argument('--enable-logging')
   chrome_options.add_argument('--log-level=0')
   chrome_options.add_argument('--v=99')
   chrome_options.add_argument('--single-process')
   chrome_options.add_argument('--data-path=/tmp/data-path')
   chrome_options.add_argument('--ignore-certificate-errors')
   chrome_options.add_argument('--homedir=/tmp')
   chrome_options.add_argument('--disk-cache-dir=/tmp/cache-dir')
   chrome_options.add_argument('user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36')
   
  driver = webdriver.Chrome(executable_path = binary_path,options=chrome_options)

These are the chrome options that are needed to set in order to make chrome work in headless mode with the latest Chrome driver version 84 — Red Gundu, Jul 30 '20 at 01:48

score 1 · Answer 10 · answered Nov 19 '20 at 12:11

System.setProperty("webdriver.chrome.driver",
         "D:\\Lib\\chrome_driver_latest\\chromedriver_win32\\chromedriver.exe");
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.addArguments("--allow-running-insecure-content");
chromeOptions.addArguments("--window-size=1920x1080");
chromeOptions.addArguments("--disable-gpu"); 
chromeOptions.setHeadless(true);
ChromeDriver driver = new ChromeDriver(chromeOptions);

score 1 · Answer 11 · answered Mar 21 '23 at 21:04

At first, I faced an issue - a blank screen while using headless mode for Chrome with Selenium.

I tried overriding user-agent, but it didn't help. And then I saw in the documentation that for Chrome versions 109 and further it recommended setting headless mode the following way:

options.addArguments("--headless=new")

https://www.selenium.dev/selenium/docs/api/java/org/openqa/selenium/chromium/ChromiumOptions.html#setHeadless(boolean)

So the full code to start headless mode is:

ChromeOptions options = new ChromeOptions()
options.addArguments("--window-size=1920,1080")
options.addArguments("--headless=new") 
options.setExperimentalOption("w3c", false)

score 1 · Answer 12 · answered Apr 26 '23 at 01:32

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')

driver = webdriver.Chrome('/usr/bin/chromedriver', options=chrome_options)
driver.get('https://www.example.com')
print(driver.title)
driver.quit()

score 0 · Answer 13 · edited Oct 06 '20 at 21:53

0

chromeoptions=add_argument("--no-sandbox");
add_argument("--ignore-certificate-errors");
add_argument("--disable-dev-shm-usage'")

is not a supported browser

solution:

Open Browser    ${event_url}    ${BROWSER}   options=add_argument("--no-sandbox"); add_argument("--ignore-certificate-errors"); add_argument("--disable-dev-shm-usage'")

don't forget to add spaces between ${BROWSER} options

edited Oct 06 '20 at 21:53

Ruli

2,592
12
30
40

answered Oct 06 '20 at 15:10

user5443130

1
1

2

Can you explain your answer? – x3l51 Oct 06 '20 at 15:47

score 0 · Answer 14 · answered Jun 28 '21 at 11:52

There is an option to hide the chromeDriver.exe window in alpha and beta versions of Selenium 4.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService # Similar thing for firefox also!
from subprocess import CREATE_NO_WINDOW # This flag will only be available in windows
chrome_service = ChromeService('chromedriver', creationflags=CREATE_NO_WINDOW)
driver = webdriver.Chrome(service=chrome_service) # No longer console window opened, niether will chromedriver output

You can check it out from here. To pip install beta or alpha versions, you can do "pip install selenium==4.0.0.a7" or "pip install selenium==4.0.0.b4" (a7 means alpha-7 and b4 means beta-4 so for other versions you want, you can modify the command.) To import a specific version of a library in python you can look here.

score 0 · Answer 15 · answered Feb 26 '23 at 12:21

RECENT UPDATE

Recently there is an update performed on headless mode of Chrome. The flag --headless is now modified and can be used as below

For Chrome version 109 and above, --headless=new flag allows us to explore full functionality Chrome browser in headless mode. For Chrome version 108 and below (till Version 96), --headless=chrome option will provide us the headless chrome browser.

So, let's add

options.add_argument("--headless=new")

for newer version of Chrome in headless mode as mentioned above.

The below works fine for me with Chrome version 110.0.5481.104

chrome_driver_path = r"E:\driver\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument('--disable-gpu')

//New Update
options.add_argument("--headless=new")  

options.binary_location = r"C:\Chrome\Application\chrome.exe"
browser = webdriver.Chrome(chrome_driver_path, options=options)
browser.get('https://www.google.com')

Walker · Answer 16 · 2023-06-04T17:53:30.157

Since executable path is depricated

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service

from selenium.webdriver import ChromeOptions
from webdriver_manager.chrome import ChromeDriverManager
from bs4 import BeautifulSoup

options = ChromeOptions()
options.add_argument('--headless=new')
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)

score -3 · Answer 17 · answered Aug 18 '21 at 18:20

-3

Update August 2021:

The fastest way to do is probably:

from selenium import webdriver  

options = webdriver.ChromeOptions()
options.set_headless = True
driver = webdriver.Chrome(options=options)

options.headless = True is deprecated.

answered Aug 18 '21 at 18:20

Pedro Burgos

3
1

didn't work for me. Needed go through `options.add_argument('headless')` – Andrew Anderson Aug 18 '21 at 18:52
This is duplicates existing answers and is incorrect since `set_headless` is a method and is actually deprecated: `DeprecationWarning: use setter for headless property instead of set_headless` so should use `options.headless` – Cas Aug 23 '21 at 18:20