How can I download a file from clicking event using Selenium on Colaboratory?

Question

I want to download files from a website.

The website has a file download button.

But when I click the download button by button.click(), the file doesn't download on Colaboratory. (It works at Juypyter Notebook.)

I googled and found a solution. My code applying the solution was as follows:

!apt-get update
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin

from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

# solution
prefs = {'download.default_directory' : "/some/path"}
options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome('chromedriver',options=options)

# Download code
xpath = '''//*[@id="glopopd_excel"]/span'''
element_get_excel = driver.find_element_by_xpath(xpath).click()

But I can't find downloaded files anywhere.

Do I have to type 'GoogleDrive path' on prefs? (But it didn't work.)

If not, how can I download files on a click event using Selenium on Colaboratory?

Answer 1

Yes, you have to type GoogleDrive path but not normally, as you expect. You have to first mount/ask permission to read and write file, using:

from google.colab import drive
drive.mount('/content/gdrive')

Run this cell first, it will ask for the google drive permission, complete this steps accordingly and now you can do anything with google drive. Now it is same as our local machine. If you want then you may change directory now, using !cd <path>. I don't know how you are trying to download file but now it is same as local machine. You can visit these file in provided directory of google drive, after download.

First completely run that cell, and run your file downloader cell. Just doing this should work for you.

You can visit here for more/detail information.

Edit:
When you are saving files, simply specify the Google Drive path for saving the file. When using large files, Colab sometimes syncs the VM and Drive asynchronously. To force the sync, simply run:

from google.colab import drive
drive.flush_and_unmount()