1

I have to download an image from the nasa website. Problem is, that my code sometimes works, sucessfully downloading an image, while sometimes saves only 186B (don't know why exactly 186).

Problems is for sure connected with the way nasa sahres those photos. For instance, an image from that link https://mars.jpl.nasa.gov/msl-raw-images/msss/00001/mcam/0001ML0000001000I1_DXXX.jpg is saved sucessfully, while from that link https://mars.nasa.gov/mer/gallery/all/2/f/001/2F126468064EDN0000P1001L0M1-BR.JPG fails.

Here is my code

public static void saveImage(String imageUrl, String destinationFile){
    URL url;
    try {
        url = new URL(imageUrl);
        System.out.println(url);
        InputStream is = url.openStream();
        OutputStream os = new FileOutputStream(destinationFile);

        byte[] b = new byte[2048];
        int length;

        while ((length = is.read(b)) != -1) {
            os.write(b, 0, length);
        }

        is.close();
        os.close();
    } catch (MalformedURLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

}

Does someone have an idea, why is doesn't work?


public boolean downloadPhotosSol(int i) throws JSONException, IOException {
    String url0 =  "https://api.nasa.gov/mars-photos/api/v1/rovers/spirit/photos?sol=" + this.chosenMarsDate + "&camera=" + this.chosenCamera + "&page=" + i + "&api_key=###";
    JSONObject json = JsonReader.readJsonFromUrl(url0);
    if(json.getJSONArray("photos").length() == 0) return true;
    String workspace = new File(".").getCanonicalPath();
    String pathToFolder = workspace+File.separator+this.getManifest().getName() + this.chosenMarsDate + this.chosenCamera +"Strona"+i;
    new File(pathToFolder).mkdirs();
    for(int j = 0;j<json.getJSONArray("photos").length();j++) {
        String url = ((JSONObject) json.getJSONArray("photos").get(j)).getString("img_src");
        SaveImage.saveImage(url, pathToFolder+File.separator+"img"+j+".jpg");
    }
    return false;
}
  • 1
    There should be byte[] b = new byte[2048]; instead of byte[] b = new byte[3]; – Szymon Maksymiuk Jan 25 '19 at 18:11
  • What is the contents of the 186b file? It is probably a server error message. – Chloe Jan 25 '19 at 18:27
  • Your code leaks resources in the event of exceptions. – Raedwald Jan 25 '19 at 18:30
  • 1
    301 Moved Permanently

    301 Moved Permanently


    CloudFront

    this is what 186b file contains

    – Szymon Maksymiuk Jan 25 '19 at 18:36
  • Please print `imageUrl` before you try to fetch it. I think something else is wrong. What is the contents of `curl -i https://mars.nasa.gov/mer/gallery/all/2/f/001/2F126468064EDN0000P1001L0M1-BR.JPG` because it is *not* redirecting for me. Are you *sure* it is the same URL? – Chloe Jan 25 '19 at 19:03
  • I've printed and it shows this: http://mars.nasa.gov/mer/gallery/all/2/f/300/2F153001277EDN89AIP1121R0M1-BR.JPG – Szymon Maksymiuk Jan 25 '19 at 19:31

2 Answers2

3

When you get a 186 byte file, open it with a text editor and see what is inside. It could contain an HTTP error message in HTML format. If instead you see the first 186 bytes of your image file, then something is not working right with your program.

EDIT: From your comments it looks like you are getting an HTTP 301 response, which is a redirect to another location. A web browser handles this automatically without you noticing. However, your Java program is not following the redirect to the new location. You need to use an HTTP Java library that handles redirects.

Sean N.
  • 963
  • 2
  • 10
  • 23
  • 2
    301 Moved Permanently

    301 Moved Permanently


    CloudFront

    186b shows that. Still I dont know why it works for one kind of links and doesnt for others i.e. don't know how to solve my problem

    – Szymon Maksymiuk Jan 25 '19 at 18:39
  • Use `curl -i http://link...` to view the `Location` header and download from the new location. Java is not following redirects like the browser is. – Chloe Jan 25 '19 at 18:46
  • Where should I do it? Cuz I'm downloading photos that had been specified by users so I need to do it in a loop – Szymon Maksymiuk Jan 25 '19 at 18:48
  • 1
    Then you have to read the HTTP response code and look at the HTTP location header and follow redirects. Try another library like https://stackoverflow.com/questions/6051648/httpclient-in-java – Chloe Jan 25 '19 at 18:50
  • Yep, what @Chloe said. Find a library to use that follows redirects. – Sean N. Jan 25 '19 at 18:57
  • CloudFront Fri, 25 Jan 2019 22:44:35 GMT text/html 183 keep-alive https://mars.nasa.gov/mer/gallery/all/2/f/300/2F153001277EDN89AIP1121L0M1-BR.JPG Redirect from cloudfront 1.1 aa9873ca0eff886ad72852b2bde57831.cloudfront.net (CloudFront) _cpfC_iVMfNY4Cte5h_3pkTh7PmPQDB-xUh9xoVd5J2SIfMtXMLvIA== Those are headers for my file that doesnt work. What should I do next? – Szymon Maksymiuk Jan 25 '19 at 22:46
  • I found a solution! In the json link wasn't secured (http), and redirection was to secured, https. I've changed my code to manually change adress and it works. So problem about URL not handling redirection stays, but my code now works. Thanks for all your help. – Szymon Maksymiuk Jan 25 '19 at 23:26
1

Best and short way of doing it:

try(InputStream in = new URL("http://example.com/image.jpg").openStream()){
    Files.copy(in, Paths.get("C:/File/To/Save/To/image.jpg"));
}
Ratish Bansal
  • 1,982
  • 1
  • 10
  • 19
  • But that doesn't answer the question, why for this link is working: https://mars.jpl.nasa.gov/msl-raw-images/msss/00001/mcam/0001ML0000001000I1_DXXX.jpg, and for this: https://mars.nasa.gov/mer/gallery/all/2/f/001/2F126468064EDN0000P1001L0M1-BR.JPG isn't – Szymon Maksymiuk Jan 25 '19 at 18:19
  • I tried the way mentioned in question and its working for me. – Ratish Bansal Jan 25 '19 at 18:27
  • Wow I didn't know you could pass parameters to `try`. What version of Java is that? – Chloe Jan 25 '19 at 18:30
  • @SzymonMaksymiuk can you please show how are you calling the saveImage method. – Ratish Bansal Jan 25 '19 at 18:30
  • @Chloe this is java 8 .To study more on this , read about try-with-resources – Ratish Bansal Jan 25 '19 at 18:31
  • 1
    @RatishBansal try-with-resources comes with java 7 https://www.oracle.com/technetwork/articles/java/trywithresources-401775.html – rkosegi Jan 25 '19 at 18:37
  • @RatishBansal Does my code work in both cases for you? It would be interesting becasue for three friends of mine and me as well doesn't. – Szymon Maksymiuk Jan 25 '19 at 18:37
  • Also the way @RatishBansal presented also doesn't work. Same problem – Szymon Maksymiuk Jan 25 '19 at 18:45
  • @SzymonMaksymiuk i am able to download both the image by executing the code shared by you.Can you please share the part of code where you are calling the saveImage method – Ratish Bansal Jan 25 '19 at 18:45
  • @RatishBansal I've posted as an answer – Szymon Maksymiuk Jan 25 '19 at 18:50
  • @SzymonMaksymiuk can you please check 2 things. 1. The url you are getting is same everytime you mentioned. 2. Are you able to open the url in browser. Also can you please run saveImage("https://mars.jpl.nasa.gov/msl-raw-images/msss/00001/mcam/0001ML0000001000I1_DXXX.jpg", "abc.jpg"); directly without any loop etc. – Ratish Bansal Jan 25 '19 at 18:59
  • Answering first and second question at once. Every time I get proper link that shows correct photo when is run in a browser. I can run, but link you've posted works in loop. https://mars.nasa.gov/mer/gallery/all/2/f/001/2F126468064EDN0000P1001L0M1-BR.JPG doesn't. Suprisingly when called directly it works now. I don't have idea why. – Szymon Maksymiuk Jan 25 '19 at 19:18
  • If it work directly with code and not in loop i think nasa might be blocking your ip after certain requests. Though i am not sure. – Ratish Bansal Jan 25 '19 at 19:22
  • It shouldn't. Link that works in the loop is also from the nasa, form the other rover. – Szymon Maksymiuk Jan 25 '19 at 19:25