Url file is not found, but in browser url works

Question

I am using Apache commons FileUtils to download a csv file and save it:

FileUtils.copyURLToFile(new URL("http://www.google.com/finance/historical?q=AAL&startdate=May+14%2C+2017&output=csv"), new File(toFilePrefix + "sadfsd.csv"), 10000, 10000);

I get the following error:

java.io.FileNotFoundException: http://www.google.com/finance/historical?q=AAL&startdate=May+14%2C+2017&output=csv

When I run the url itself in a browser it works and the file is downloaded.

If I change the url slighty to get Google's stock it works:

http://www.google.com/finance/historical?q=GOOG&startdate=May+14%2C+2017&output=csv

Entire Stacktrace:

Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
    Mon May 29 19:28:02 PDT 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    Mysql Connection has been established!
    MySQL JDBC Driver has been loaded ...
    java.io.FileNotFoundException: http://www.google.com/finance/historical?q=AAL&startdate=Mar+14+2017&output=csv
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1836)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
        at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1506)
        at com.mycompany.myapp.AppFiles.downloadFiles(AppFiles.java:61)
        at com.mycompany.myapp.Main.main(Main.java:8)
    ------------------------------------------------------------------------
    BUILD SUCCESS
    ------------------------------------------------------------------------
    Total time: 0.980s
    Finished at: Mon May 29 19:28:03 PDT 2017
    Final Memory: 5M/245M
    ------------------------------------------------------------------------

I have also tried downloading this file using the accepted answer from this question (Using Java NIO) and the same error appears: How to download and save a file from Internet using Java?.

Your code worked for me. No exceptions seen! Could be an issue with `new File(toFilePrefix + "sadfsd.csv")` ? — harshavmb, May 30 '17 at 02:29
@harshavmb Strange. It is not working for me running inside of netbeans using Java 8. Other URL's seem to work, but this one doesn't. I do not think it is an issue with the new File constructor because it saves other csv files. — user2924127, May 30 '17 at 02:34
I have tired this code in both Eclipse and Netbeans without any luck. — user2924127, May 30 '17 at 02:52
Did you copy/paste the url or type it in? I have had strange issues on some pastes. — KevinO, May 30 '17 at 02:55
I'm starting to suspect that the string passed to `new URL` may have some non-printing characters in it. Move the String to a separate assignment and print its length. — VGR, May 30 '17 at 02:56
okay, I guess your app is receiving `404` response while hitting the above url. Can you add these snippets before `FileUtils.copyURLToFile` ? `URL url = new URL("http://www.google.com/finance/historical?q=AAL&startdate=Mar+14+2017&output=csv"); HttpURLConnection connection = (HttpURLConnection)url.openConnection(); connection.setRequestMethod("GET"); connection.connect(); System.out.println(connection.getResponseCode());` — harshavmb, May 30 '17 at 02:57
@harshavmb It prints 200. So it seems the response code is okay — user2924127, May 30 '17 at 03:01
@VGR The string length is 86 when I go : String str = "http://www.google.com/finance/historical?q=AAL&startda‌te=May+14%2C+2017&ou‌tput=csv"; System.out.println(str.length());. By my count there should only be 80???? — user2924127, May 30 '17 at 03:02
Instead of copy-and-pasting your URL into your code, take the time to type it in by hand. That should eliminate the non-printing characters. — VGR, May 30 '17 at 03:05
@VGR When I type it and then go length it shows 79 which is correct, but the error still shows. — user2924127, May 30 '17 at 03:14
It really seems to be a problem with the character AAL in the url. If I change this to almost anything then it works. As soon as I put in AAL it stops — user2924127, May 30 '17 at 03:36
The length of your URL should be 82. You can expose the nonprinting characters with something like: `String str = "http://www.google.com/finance/historical?q=A\u200dAL&startdate=May+14%2C+2017&output=csv"; str.chars().filter(c -> c > 126).forEach(c -> System.out.printf("%02x ", c)); System.out.println();` — VGR, May 30 '17 at 11:45

Url file is not found, but in browser url works

0 Answers0