Python: how to download .csv files using Selenium?

Question

I am using Selenium for navigating the following website:

https://apps1.eere.energy.gov/sled/#/

I would like to have data for a city like Boston: what I am doing is the following:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

driver = webdriver.Firefox()
driver.get("https://apps1.eere.energy.gov/sled/#/")
search = driver.find_element_by_class_name('sr-only')
search.send_keys("Boston")
search.send_keys(Keys.RETURN)
time.sleep(5)
tmp = driver.find_element_by_css_selector("a.csv-link")

This methodology allow me to arrive at the following webpage:

https://apps1.eere.energy.gov/sled/#/results/elecandgas?city=Boston&abv=MA&section=electricity&currentState=Massachusetts&lat=42.3600825&lng=-71.05888010000001

Now I would like to download a csv file under State and National Retail Electricity Rate Trends by clicking on Download Chart and then Download Chart\Data CSV.

I try to click on it:

tmp.click()

but I got the error:

ElementNotInteractableException: Message:

I report in the inspecting the button (Download Chart/Data) I would like to click by showing the screenshot of the inspection element in Firefox.

try finding the element by a different method. `XPath` is usually pretty reliable. — SuperStew, Jan 23 '18 at 14:59
`a.csv-link` is not a class-name. (It includes a class-name.) — Andy G, Jan 23 '18 at 15:00

score 1 · Accepted Answer · answered Jan 23 '18 at 15:34

1

There are two steps involved to download the file after reaching the page you specified. First you have to click on the dorp-down button then you have to select the csv data option. Using the corresponding xpath for these elements,

driver.find_element_by_xpath('//*[@id="historical-electric-download"]/a[@class="no-underline"]').click()
driver.find_element_by_xpath('//*[@id="historical-electric-download"]/div/a[@class="csv-link"]').click()

answered Jan 23 '18 at 15:34

hashmuke

3,075
2
18
29

Do you know how I can automatically save the file? – emax Jan 23 '18 at 15:44
For that I suggest getting the direct link to the file then checkout the following question [How do I download a file over HTTP using Python? ](https://stackoverflow.com/questions/22676/how-do-i-download-a-file-over-http-using-python) – hashmuke Jan 23 '18 at 15:53

score 0 · Answer 2 · answered Jan 23 '18 at 15:04

0

'a.csv-link' is not class. You can try using id which is present in page.

You can try this:

content = driver.find_element_by_id('csv-link')

answered Jan 23 '18 at 15:04

Dipak Pawar

71
1
3

score 0 · Answer 3 · answered Jan 23 '18 at 15:10

You are trying to click on Download chat first and then you are trying to click on Download Chart/Data CSV to download the CSV file , and the problem is with the locator you are using

You are trying to click on the element based on class name , but the locator doesn't have class attribute.

So try with the below xpath after clicking on download chat.

//a[@id='csv-link']

score 0 · Answer 4 · answered Jan 23 '18 at 15:15

0

a.csv-link is a css locator, so you need driver.find_element_by_css_selector

answered Jan 23 '18 at 15:15

jf328

6,841
10
58
82

1

The OP is already using `driver.find_element_by_css_selector` by the look of their code, aren't they? – Vince Bowdren Jan 23 '18 at 16:39
@VinceBowdren, that was a later edit. Earlier he's using `find_element_by_class_name`. You can see lots of other answers/comments are referring to that point as well – jf328 Jan 24 '18 at 09:46

Python: how to download .csv files using Selenium?

4 Answers4