Selenium-python downloading but file is saved as .part

Question

My script works but it's saving the file as a .part, although checking this against a manually downloaded file its the same size and thankfully complete. I can't understand why it's being saved as a partial file though. Sorta inconvenient for my next idea. Does anybody have an idea of why this might be? Here's my code...which works...

from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
import time
import mechanize
import urllib
from urllib import urlretrieve

fp = webdriver.FirefoxProfile()

fp.set_preference("browser.download.folderList",1)
fp.set_preference("browser.download.manager.showWhenStarting",False)
fp.set_preference("browser.download.dir",'Users/matthewyoung/Downloads')
fp.set_preference("browser.helperApps.neverAsk.saveToDisk","Plain text")
fp.set_preference("browser.download.manager.scanWhenDone",False)
fp.set_preference("browser.download.manager.showAlertOnComplete",True)
fp.set_preference("browser.download.manager.useWindow",False)
fp.set_preference("browser.helperApps.alwaysAsk.force",False)

browser = webdriver.Firefox(firefox_profile=fp)



#browser = webdriver.Firefox() # Get local session of firefox
browser.get("http://vizier.u-strasbg.fr/vizier/surveys.htx") # Load page
assert "VizieR" in browser.title
#p = raw_input('Star name? ')
elem = browser.find_element_by_name('-c') # Find the query box
elem.send_keys('mwc 560' + Keys.RETURN)
time.sleep(0.2) # Let the page load, will be added to the API
elem=browser.find_element_by_name('-out.max')
elem.send_keys('unlimited'+Keys.TAB)
elem2=browser.find_element_by_name('-out.form')
time.sleep(0.5)
elem2.send_keys('; -Separated-Values')
time.sleep(0.5)
elem2.send_keys(Keys.TAB)
elem2.send_keys(Keys.TAB)
time.sleep(0.2)
browser.find_element_by_class_name('data').submit()
time.sleep(3.0)
#df=elem2.send_keys(Keys.SPACE)
#print df
browser.close()

Hmmm...that's interesting. That's what I was expecting. I'm running it out of Eclipse. I'm gonna try somewhere else, see if that makes any difference. — Matt, Jul 01 '13 at 01:15
Are you hitting the popup window manually? Cuz I'm trying to avoid that. — Matt, Jul 01 '13 at 01:21
i changed to `webdriver.Chrome()` and i dint press aything while it was running and it worked fine — Serial, Jul 01 '13 at 01:24
That's good to know because I'd like to stay away from Chrome as this project is going to be used on a 'firefox only' computer. — Matt, Jul 01 '13 at 01:34
@Matt - Did you find a solution for this problem? I am trying to save .CSV and .Zip files from Firefox. FirefoxProfile creates the default download dir, but saves the files with a .exe.part or .csv.part extension. The 'SaveToDisk' popup window is still displayed when I run the test. — nids, Dec 05 '13 at 21:26
@nids As far as I know there is no way to fix this issue with Selenium. All these months later and I'm still trying to figure out a clean way of downloading files in a similar manner. The 'hack' way I figured out was to identify where the files are going and set up the script further to rename them. My files (to my recollection) were fully downloaded, but kept getting that `.part` added to the end of their names. — Matt, Dec 05 '13 at 22:07

score 3 · Answer 1 · answered Oct 17 '13 at 15:28

It is downloading as .part because that popup save as dialog window appears. Python cannot deal with the popup window. I have found that when you try to set settings for a custom profile in webdriver it doesn't necessarily work (for instance I was able to set a custom profile in selenium to download a csv but not a pdf). However, I was able to solve my pdf problem by creating a custom profile in firefox. I am not very experienced with tsv files so I am not sure what setting that would be. If you can create a new firefox profile (following the instructions here: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles) you can try to set that profile to save tsv by default. If you don't know the exact setting to go in and change in "about:config" you can try just click the checkbox on the popup to always save those kinds of files.

From there you set your profile to that custom profile you created like this:

    profile = webdriver.firefox.firefox_profile.FirefoxProfile("/Users/matthewyoung/Library/Application Support/Firefox/Profiles/"YOUR PROFILE NAME")

Keep in mind that YOUR PROFILE NAME will have a bunch of random letters first, so follow that path to find the actual profile name.

score 0 · Answer 2 · answered Feb 19 '14 at 05:10

I think the only thing you are missing from your Firefox profile setting is the following

fp.set_preference("browser.helperApps.neverAsk.openFile",
                       'Plain Text')

So the entire code should be

from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
import time

fp = webdriver.FirefoxProfile()

fp.set_preference("browser.download.folderList",2)
fp.set_preference("browser.download.manager.showWhenStarting",False)
fp.set_preference("browser.download.dir",'Users/matthewyoung/Downloads')

fp.set_preference("browser.helperApps.neverAsk.openFile", 'Plain Text')
fp.set_preference("browser.helperApps.neverAsk.saveToDisk","Plain text")
fp.set_preference("browser.download.manager.scanWhenDone",False)
fp.set_preference("browser.download.manager.showAlertOnComplete",True)
fp.set_preference("browser.download.manager.useWindow",False)
fp.set_preference("browser.helperApps.alwaysAsk.force",False)

browser = webdriver.Firefox(firefox_profile=fp)


browser.get("http://vizier.u-strasbg.fr/vizier/surveys.htx") # Load page
assert "VizieR" in browser.title

elem = browser.find_element_by_name('-c') # Find the query box
elem.send_keys('mwc 560' + Keys.RETURN)
time.sleep(0.2) # Let the page load, will be added to the API
elem=browser.find_element_by_name('-out.max')
elem.send_keys('unlimited'+Keys.TAB)
elem2=browser.find_element_by_name('-out.form')
time.sleep(0.5)
elem2.send_keys('; -Separated-Values')
time.sleep(0.5)
elem2.send_keys(Keys.TAB)
elem2.send_keys(Keys.TAB)
time.sleep(0.2)
browser.find_element_by_class_name('data').submit()
time.sleep(3.0)

browser.close()

score 0 · Answer 3 · edited Nov 21 '14 at 12:34

0

The following value should be used for plain text:

fp.set_preference("browser.helperApps.neverAsk.saveToDisk","text/plain")

edited Nov 21 '14 at 12:34

Bowdzone

3,827
11
39
52

answered Nov 21 '14 at 11:56

user4278453

1

Selenium-python downloading but file is saved as .part

3 Answers3

Linked