How to use chrome webdriver in selenium to download files in python?

Question

Based on the posts here and here I am trying to use a chrome webdriver in selenium to be able to download a file. Here is the code so far

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
chrome_options.add_experimental_option("profile.default_content_settings.popups", 0)
chrome_options.add_experimental_option("download.prompt_for_download", "false")
chrome_options.add_experimental_option("download.default_directory", "/tmp")

driver = webdriver.Chrome(chrome_options=chrome_options)

But this alone results in the following error:

WebDriverException: Message: unknown error: cannot parse capability: chromeOptions
from unknown error: unrecognized chrome option: download.default_directory
  (Driver info: chromedriver=2.24.417424 (c5c5ea873213ee72e3d0929b47482681555340c3),platform=Linux 4.10.0-37-generic x86_64)

So how to fix this? Do I have to use this 'capability' thing? If so, how exactly?

score 42 · Accepted Answer · answered Oct 25 '17 at 17:07

42

Try this. Executed on windows

(How to control the download of files with Selenium Python bindings in Chrome)

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_experimental_option("prefs", {
  "download.default_directory": r"C:\Users\xxx\downloads\Test",
  "download.prompt_for_download": False,
  "download.directory_upgrade": True,
  "safebrowsing.enabled": True
})

answered Oct 25 '17 at 17:07

Satish

1,976
1
15
19

3

Yes thats it... I used the options in the wrong way. Hard to get the right documentation. Is that somewhere documented, except SO...? – Alex Oct 26 '17 at 12:26
11

if you would care to explain what is the purpose of setting safe browsing and what is directory upgrade please? – Dkyc Jun 01 '18 at 00:41
6

Ensure `driver = webdriver.Chrome(chrome_options=chrome_options)` and not just `driver = webdriver.Chrome()` – Ivan Chau Jun 28 '19 at 03:11
This doesn't seem to be reliable... I've started getting download prompt to save the file.. would appreciate any pointers! – Jay Jan 25 '21 at 15:22
Best guess; this doesn't work if the employer managed chrome does not let me set these settings manually. :( – Jay Jan 25 '21 at 16:25
@Alex - You can find good documentation [here](https://peter.sh/experiments/chromium-command-line-switches/#condition-13) – Shreyesh Desai Dec 21 '21 at 07:56
@Dkyc I've found the best documentation on available prefs and what they're supposed to do to be the Chromium source code, under chrome/common/pref_names.cc. E.g., for `directory_upgrade`: // Boolean that records if the download directory was changed by an // upgrade a unsafe location to a safe location. const char kDownloadDirUpgraded[] = "download.directory_upgrade"; So I'd say it shouldn't be useful. – Apteryx Jan 21 '22 at 18:57
1

And indeed, only `download.default_directory` and `download.prompt_for_download` are required, as of Chromium 97. – Apteryx Jan 21 '22 at 19:01
https://source.chromium.org/chromium/chromium/src/+/main:chrome/common/pref_names.cc also relevant – robd Sep 27 '22 at 10:20
Just to say that it works well on linux also – floupinette Dec 12 '22 at 21:07

Andrey Semakin · Answer 2 · 2019-03-24T10:45:06.070

I think that the easiest way to save arbitrary file (i.e. image) using WebDriver is to execute JavaScript which will save file. No configuration required at all!

I use this library FileSaver.js to save a file with desired name with ease.

from selenium import webdriver
import requests

FILE_SAVER_MIN_JS_URL = "https://raw.githubusercontent.com/eligrey/FileSaver.js/master/dist/FileSaver.min.js"

file_saver_min_js = requests.get(FILE_SAVER_MIN_JS_URL).content

chrome_options = webdriver.ChromeOptions()
driver = webdriver.Chrome('/usr/local/bin/chromedriver', options=chrome_options)

# Execute FileSaver.js in page's context
driver.execute_script(file_saver_min_js)

# Now you can use saveAs() function
download_script = f'''
    return fetch('https://cdn.sstatic.net/Sites/stackoverflow/company/img/logos/so/so-logo.svg?v=a010291124bf',
        {{
            "credentials": "same-origin",
            "headers": {{"accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-language":"en-US,en;q=0.9"}},
            "referrerPolicy": "no-referrer-when-downgrade",
            "body": null,
            "method": "GET",
            "mode": "cors"
        }}
    ).then(resp => {{
        return resp.blob();
    }}).then(blob => {{
        saveAs(blob, 'stackoverflow_logo.svg');
    }});
    '''

driver.execute_script(download_script)
# Done! Your browser has saved an SVG image!

To bypass repeating downloading file warnings or WebDriver Server (great for docker), use `driver.execute_script return readAsDataURL the with FileReader` — Kelvin Ng, May 09 '19 at 13:52

score 5 · Answer 3 · answered Jun 17 '18 at 04:26

Some tips:

chromium and chromedriver should have same version.

Typically chromium package should have chromedriver inside, you can find it in the install dir. If you are using ubuntu/debian, execute dpkg -L chromium-chromedriver.
Have a correct Chrome preference config.

as Satish said, use options.add_experimental_option("prefs", ...) to config selenium+chrome. But sometimes the config may change by time. The beset way to get newest and workable prefs is to check it in the chromium config dir. For example,
- Launch a chromium in Xorg desktop
- Change settings in menu
- Quit chromium
- Find out the real settings in ~/.config/chromium/Default/Preferences
- Read it, pick out the exact options you need.

In my case, the code is:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

options = webdriver.ChromeOptions()
options.gpu = False
options.headless = True
options.add_experimental_option("prefs", {
    "download.default_directory" : "/data/books/chrome/",
    'profile.default_content_setting_values.automatic_downloads': 2,
    })

desired = options.to_capabilities()
desired['loggingPrefs'] = { 'performance': 'ALL'}
driver = webdriver.Chrome(desired_capabilities=desired)

where is chromium config dir? – Mostafa Dec 28 '18 at 18:16 — Mostafa, Dec 28 '18 at 18:16

score 2 · Answer 4 · answered Jan 24 '20 at 16:30

2

for chrome in mac os, the download.defaultdirectory did not work for me and fortunately savefile.default_directory works.

prefs = {
    "printing.print_preview_sticky_settings.appState": json.dumps(settings),
    "savefile.default_directory": "/Users/creative/python-apps",
    "download.prompt_for_download": False,
    "download.directory_upgrade": True,
    "download.safebrowsing.enabled": True
}

answered Jan 24 '20 at 16:30

neelmeg

2,459
6
34
46

Only `download.default_directory` and `download.prompt_for_download` are required, as of Chromium 97. – Apteryx Jan 21 '22 at 19:02

score 1 · Answer 5 · answered Nov 15 '20 at 14:13

One of the reasons you can't set "download.default_directory" may be that you have a system variable XDG_DOWNLOAD_DIR in file ~/.config/user-dirs.dirs

You can remove variable form that file or, you can set it to whatever you like before running your program.

I was looking for a solution for two days...

My SW set:

Ubuntu bionic, 18.04.5 LTS
chromedriver.86.0.4240.22.lin64
Python 3.9
selenium 3.141.0
splinter 0.14.0

score 0 · Answer 6 · answered Oct 25 '17 at 18:19

From your exception, you are using chromedriver=2.24.417424.

What are the versions of Selenium and Chrome browser that are you using?

I tried the following code with:

Selenium 3.6.0
chromedriver 2.33
Google Chrome 62.0.3202.62 (Official Build) (64-bit)

And it works:

from selenium import webdriver

download_dir = "/pathToDownloadDir"
chrome_options = webdriver.ChromeOptions()
preferences = {"download.default_directory": download_dir ,
               "directory_upgrade": True,
               "safebrowsing.enabled": True }
chrome_options.add_experimental_option("prefs", preferences)
driver = webdriver.Chrome(chrome_options=chrome_options,executable_path=r'/pathTo/chromedriver')

driver.get("urlFileToDownload");

Make sure you are using a browser that is supported by your chromedriver (from here, should be Chrome v52-54).

score -1 · Answer 7 · answered Jun 15 '20 at 18:36

Can't you use requests lib?

If so, here's an example:

import re
import requests

urls = [ '...' ]

for url in urls:
  # verify = False ==> for HTTPS requests without SSL certificates
  r = requests.get( url, allow_redirects = True, verify = False )

  cd = r.headers.get( 'content-disposition' )
  fa = re.findall( 'filename=(.+)', cd )

  if len( fa ) == 0:
    print( f'Error message: {link}' )
    continue

  filename = fa[ 0 ]

  f = open( os.path.join( 'desired_path', filename ), 'wb' )
  f.write( r.content )
  f.close()

How to use chrome webdriver in selenium to download files in python?

7 Answers7

Linked