6

I am web scraping with selenium and whenever i try to download i file the firefox download/save as file pops up however, even If i apply profile.set_preference('browser.helperApps.neverAsk.saveToDisk', "application/csv"), it still doesnt work, I have tried everyt .csv related MIME but doesn't work, is it possible to either click save as radio button and then click ok on the dialog or disable it entirely.

  • 1
    Did you also set `browser = webdriver.Firefox(firefox_profile=profile)` as [suggested here](http://stackoverflow.com/questions/1176348/access-to-file-download-dialog-in-firefox/7983487#7983487)? – unutbu Aug 27 '12 at 17:33
  • is that a system firefox profile you edited? depending on your selenium version it might be using its own profile indepent of your system's. you may want to consider creating a custom profile for this and specifying the path to the profile using the -firefoxProfileTemplate flag when starting selenium via the command line – Will Ayd Aug 27 '12 at 17:52
  • Yes, I have tried that already. –  Aug 27 '12 at 17:56

3 Answers3

3

you should do two things, first set these three preferences as follows (this is in Java but I guess you manage to translate that to python :-):

profile.setPreference("browser.download.dir", "c:/yourDownloadDir");
profile.setPreference("browser.download.folderList", 2);
profile.setPreference("browser.helperApps.neverAsk.saveToDisk", "application/csv, text/csv");

secondly, you should make sure the download file has the desired mime type. To do that, you can use the web developer tools and inspect the download.

EDIT:

To find out the MIME type open Chrome, press Ctrl+Shift+I (Cmd+Alt+I on Mac OS) change to the 'Network' tab and click your download link. You should see something like this:

Chrome developer tools mime type

Jeremy Thompson
  • 61,933
  • 36
  • 195
  • 321
Tim Büthe
  • 62,884
  • 17
  • 92
  • 129
  • I have already tried that, when I download it manually it says it is a Microsoft Excell Comma Seperated Value, however the MIME type is not displayed in the browsers options-applications. This is the javascript hyperlink ` href="javascript:download_as('CSV');` –  Aug 28 '12 at 13:52
  • 2
    Is there a way to allow everything to download in selenium? –  Aug 28 '12 at 15:57
  • @user1582983 I added an explanation how to find out the mime type. – Tim Büthe Aug 29 '12 at 08:42
  • That doesn't work, it is javascript and does not show here the file is located. –  Aug 29 '12 at 13:51
  • You don't need to know where the file is located. If it gets downloaded by the browser, you'll see it in the network tab as shown on the screenshot. Just open the tab, click the link so the download is invoked and find the entry. – Tim Büthe Aug 29 '12 at 13:55
  • It doesn't list the download, i dont believe there is a MIME formatted for the file. –  Aug 29 '12 at 14:03
  • If the file is send from the server to the browser, there must be an entry under network. Is this public available? Can I try it out? Can you show a screenshot of the network tab after downloading the file? – Tim Büthe Aug 29 '12 at 14:32
  • This is the website, https://mbsdisclosure.fanniemae.com/PoolTalk2/index.html, go to advanced search/press search/and press the csv image. –  Aug 29 '12 at 16:24
  • 1
    I think it is "application/vnd.csv", but you were right, it was a little hard to find out since it opens in a new tab. Never saw this MIME type before, maybe it's not a standard one and should better be changed..? – Tim Büthe Aug 30 '12 at 09:31
2

Just an additional answer that might help someone, as comments to the accepted answer put me on the right track (thanks!). Another MIME type of CSV you might be dealing with is application/x-csv - that was my case and once I looked it up in the Network tab of the browser, I became a happier man :)

almondandapricot
  • 301
  • 3
  • 9
  • Didn't work for me but the fact you shared a solution that worked for you 4 years after the question was asked is worth an upvote IMO. – Jeremy Thompson Sep 12 '16 at 07:59
0

In C#

FirefoxOptions options = new FirefoxOptions();
options.SetPreference("browser.download.folderList", 2);
options.SetPreference("browser.download.manager.showWhenStarting", false);
options.SetPreference("browser.download.dir", "c:\\temp");
options.SetPreference("browser.helperApps.neverAsk.saveToDisk", "text/csv");
Sebastian D'Agostino
  • 1,575
  • 2
  • 27
  • 44
Upraj
  • 11