7

I've been looking for a solution to do headless testing in osx. But I need the ability to save files returned by the server.

I've tested selenium, phantomjs, casperjs and have looked into anything I could find online.

none of them supports downloading. am I missing something? are there any headless browser/testing frameworks that support downloads?

Mars
  • 4,197
  • 11
  • 39
  • 63
  • CasperJS has a `download` function, so it supports downloading. PhantomJS supports downloading because CasperJS is built on top of it. What is the problem that you're having? – Artjom B. Jan 14 '15 at 11:55
  • @ArtjomB. what do you mean download function? the file is a result of an http post so it's sent as a Content-Disposition: attachment; filename=FILENAME in the header – Mars Jan 14 '15 at 13:07
  • Yes, it's true that PhantomJS isn't saving a file, when it receives such a request, but the file in question can be [downloaded](http://docs.casperjs.org/en/latest/modules/casper.html#download) such a case from an [matching event handler](http://docs.casperjs.org/en/latest/events-filters.html#resource-received). – Artjom B. Jan 14 '15 at 13:18
  • possible duplicate of [downloading a file that comes as an attachment in a POST request response in PhantomJs](http://stackoverflow.com/questions/16144252/downloading-a-file-that-comes-as-an-attachment-in-a-post-request-response-in-pha) – Artjom B. Jan 14 '15 at 13:20
  • yes! but you cannot call resource.url on a header attachment. @ArtjomB. because the resource url is just an html page. and thus not the file that needs to be saved – Mars Jan 14 '15 at 14:33

2 Answers2

4

What you can do is:

  • start a virtual display (see Xvfb)
  • start up Firefox browser with preferences configured to automatically save csv files

Working example in python with additional comments (using pyvirtualdisplay xvfb wrapper):

from os import getcwd
import time

from pyvirtualdisplay import Display
from selenium import webdriver

# start the virtual display
display = Display(visible=0, size=(800, 600))
display.start()

# configure firefox profile to automatically save csv files in the current directory
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.download.dir", getcwd())
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv")

browser = webdriver.Firefox(firefox_profile=fp)
browser.get('http://www.nationale-loterij.be/nl/onze-spelen/lotto/resultaten')

# check the option
browser.find_element_by_id('corporatebody_3_corporategrid93961a8f9b424ed6bd0697df356d9483_1_rblType_0').click()

# click the link
browser.find_element_by_name('corporatebody_3$corporategrid93961a8f9b424ed6bd0697df356d9483_1$btnDownload').click()

# hardcoded delay for waiting a file download (better check for the downloaded file to appear on the disk)
time.sleep(2)

# quit the browser
browser.quit()

# stop the display
display.stop()

See also:

Community
  • 1
  • 1
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • this does indeed download the file, but it still shows a firefox frame, so this might work but it's not really headless. or is it supposed to show firefox? – Mars Jan 20 '15 at 16:00
  • @Mars I suspect this is because you have a real display and the virtual one is not used because of it. – alecxe Jan 20 '15 at 17:43
  • Nice solution but when I attempt to do ```clear()``` on an input element I get a selenium exception ```Element is not currently interactable and may not be manipulated```. Seems odd if you could ```click()``` elements but not ```clear()``` them. – Chuck Claunch Mar 11 '17 at 02:19
  • the question specified OSX... Xvfb works for X11, not Macs. – Corey Goldberg Jun 25 '17 at 19:38
1

I use on OSX selenium + wget command to perform downloads.

Here it is a sample of code:

new_driver = webdriver.Firefox()
new_driver.get(url)
for element in new_driver.find_elements_by_tag_name('img'):
    os.system('wget ' + element.get_attribute('src').rstrip('\n'))
aberna
  • 5,594
  • 2
  • 28
  • 33
  • 2
    I can't do a wget because the file is inside a post header as Content-Disposition: attachment; filename=FILENAME – Mars Jan 14 '15 at 13:08
  • @Mars do you mind sharing the url and the code you are working on ? – aberna Jan 15 '15 at 07:54
  • @abema here's the url I'm working on http://www.nationale-loterij.be/nl/onze-spelen/lotto/resultaten click the first bullet and click download... – Mars Jan 15 '15 at 10:17