Not able to download .csv file through Google Chrome in headless mode

Question

I have been trying to download a csv file using selenium in Chrome headless mode but it is not working for my scenario. I tried to run the same script without headless and it works. I tried the same headless configuration with https://www.mockaroo.com/ site and it worked. Below is the chrome headless configuration that I'm using

System.setProperty("webdriver.chrome.driver", Constants.CHROME_DRIVER);
ChromeOptions options = new ChromeOptions();
options.addArguments("--test-type");
options.addArguments("--headless");
options.addArguments("--disable-extensions"); //to disable browser extension popup
options.addArguments("window-size=1980,1080");
options.addArguments("--disable-web-security");
options.addArguments("--no-sandbox");
options.addArguments("--disable-gpu");
options.addArguments("--disable-popup-blocking");
options.addArguments("--start-maximized");
options.addArguments("--allow-running-insecure-content");
options.setExperimentalOption("w3c", false);
options.addArguments("--ignore-certificate-errors");
options.addArguments("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36");

DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(ChromeOptions.CAPABILITY, options);

// set performance logger
// this sends Network.enable to chromedriver
LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
cap.setCapability("goog:loggingPrefs", logPrefs);

ChromeDriverService driverService = ChromeDriverService.createDefaultService();
driver = new ChromeDriver(driverService, cap);
String downloadFilepath = "C:\\Users\\User\\Downloads";
System.out.println(downloadFilepath);
Map < String, Object > commandParams = new HashMap<String, Object>();
commandParams.put("cmd", "Page.setDownloadBehavior");
Map < String, Object > params = new HashMap<String, Object>();
params.put("behavior", "allow");
params.put("download.prompt_for_download", false);
params.put("downloadPath", downloadFilepath);
commandParams.put("params", params);
ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();
try {
    String command = objectMapper.writeValueAsString(commandParams);
    String u = driverService.getUrl().toString() + "/session/" + ((ChromeDriver)driver).getSessionId() + "/chromium/send_command";
    System.out.println("U: " + u.toString());
    HttpPost request = new HttpPost(u);
    request.addHeader("content-type", "application/json");
    request.setEntity(new StringEntity(command));
    httpClient.execute(request);
} catch (Exception e) { }

driver.get(prop.getProperty("url"));

With the above driver configuration (after authenticating the user), I'm clicking a button which submits the form data and in response, I am getting:

Content-Disposition: attachment;filename=WIS_mop_mis3.csv

which downloads the file.

Please find the logs below where you can see the highlighted part where it says that net::ERR_ABORTED but I am getting this error for both headless & without headless;

Performance logs for headless mode

Anyone have any idea, where I am doing wrong.

Edit: The application flow is when user clicks on the export button a new window gets opened where user fills the data and clicks on a button which calls a function for some sort of validation. After validation, it submits the form using document.formName.submit().

Headless does not support this. Neither does is support access to the clipboard. — Jortega, Feb 13 '20 at 14:30
@Jortega headless chrome supports file download as I have tested this with some other websites. The problem is with this particular web page. — Abhishek Jha, Feb 13 '20 at 14:50
What about outside of selenium. Does it work when you try to download it manually? — Jortega, Feb 13 '20 at 15:58
Yes it works for both manual download and using chrome drive (without headless) — Abhishek Jha, Feb 13 '20 at 18:18

score 0 · Answer 1 · answered Feb 14 '20 at 14:44

I am able to figure out the solution. Below is the working code

System.setProperty("webdriver.chrome.driver", Constants.CHROME_DRIVER);
ChromeOptions options = new ChromeOptions();
options.addArguments("--test-type");
options.addArguments("--headless");
options.addArguments("--disable-extensions"); //to disable browser extension popup
options.addArguments("window-size=1980,1080");
options.addArguments("--disable-web-security");
options.addArguments("--no-sandbox");
options.addArguments("--disable-gpu");
options.addArguments("--disable-popup-blocking");
options.addArguments("--start-maximized");
options.addArguments("--allow-running-insecure-content");
options.setExperimentalOption("w3c", false);
options.addArguments("--ignore-certificate-errors");
options.addArguments("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36");

HashMap<String, Object> chromePreferences = new HashMap<String, Object>();

// This part was missing in my old code.
chromePreferences.put("download.prompt_for_download", false);
chromePreferences.put("download.directory_upgrade", true);
chromePreferences.put("safebrowsing.enabled", false);
chromePreferences.put("profile.default_content_settings.popups", 0);
chromePreferences.put("safebrowsing.disable_download_protection", true);
chromePreferences.put("download.default_directory", downloadFilepath);
options.setExperimentalOption("prefs", chromePreferences);

DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(ChromeOptions.CAPABILITY, options);

// set performance logger
// this sends Network.enable to chromedriver
LoggingPreferences logPrefs = new LoggingPreferences();
logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
cap.setCapability("goog:loggingPrefs", logPrefs);

ChromeDriverService driverService = ChromeDriverService.createDefaultService();
driver = new ChromeDriver(driverService, cap);
String downloadFilepath = "C:\\Users\\User\\Downloads";
System.out.println(downloadFilepath);
Map < String, Object > commandParams = new HashMap<String, Object>();
commandParams.put("cmd", "Page.setDownloadBehavior");
Map < String, Object > params = new HashMap<String, Object>();
params.put("behavior", "allow");
params.put("downloadPath", downloadFilepath);
commandParams.put("params", params);
ObjectMapper objectMapper = new ObjectMapper();
HttpClient httpClient = HttpClientBuilder.create().build();
try {
    String command = objectMapper.writeValueAsString(commandParams);
    String u = driverService.getUrl().toString() + "/session/" + ((ChromeDriver)driver).getSessionId() + "/chromium/send_command";
    System.out.println("U: " + u.toString());
    HttpPost request = new HttpPost(u);
    request.addHeader("content-type", "application/json");
    request.setEntity(new StringEntity(command));
    httpClient.execute(request);
} catch (Exception e) { }

This post helped me to fix this issue https://stackoverflow.com/a/52384942/6017332

Not able to download .csv file through Google Chrome in headless mode

1 Answers1