java.net.URL read stream to byte[]

Question

I'm trying to read an image from an URL (with the Java package java.net.URL) to a byte[]. "Everything" works fine, except that the content isn't being entirely read from the stream (the image is corrupt, it doesn't contain all the image data)... The byte array is being persisted in a database (BLOB). I really don't know what the correct approach is, maybe you can give me a tip. :)

This is my first approach (code formatted, removed unnecessary information...):

URL u = new URL("http://localhost:8080/images/anImage.jpg");
int contentLength = u.openConnection().getContentLength();
Inputstream openStream = u.openStream();
byte[] binaryData = new byte[contentLength];
openStream.read(binaryData);
openStream.close();

My second approach was this one (as you'll see the contentlength is being fetched another way):

URL u = new URL(content);
openStream = u.openStream();
int contentLength = openStream.available();
byte[] binaryData = new byte[contentLength];
openStream.read(binaryData);
openStream.close();

Both of the code result in a corrupted image... I already read this post from Stack Overflow.

Answer 1

There's no guarantee that the content length you're provided is actually correct. Try something akin to the following:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
InputStream is = null;
try {
  is = url.openStream ();
  byte[] byteChunk = new byte[4096]; // Or whatever size you want to read in at a time.
  int n;

  while ( (n = is.read(byteChunk)) > 0 ) {
    baos.write(byteChunk, 0, n);
  }
}
catch (IOException e) {
  System.err.printf ("Failed while reading bytes from %s: %s", url.toExternalForm(), e.getMessage());
  e.printStackTrace ();
  // Perform any other exception handling that's appropriate.
}
finally {
  if (is != null) { is.close(); }
}

You'll then have the image data in baos, from which you can get a byte array by calling baos.toByteArray().

This code is untested (I just wrote it in the answer box), but it's a reasonably close approximation to what I think you're after.

Answer 2

Just extending Barnards's answer with commons-io. Separate answer because I can not format code in comments.

InputStream is = null;
try {
  is = url.openStream ();
  byte[] imageBytes = IOUtils.toByteArray(is);
}
catch (IOException e) {
  System.err.printf ("Failed while reading bytes from %s: %s", url.toExternalForm(), e.getMessage());
  e.printStackTrace ();
  // Perform any other exception handling that's appropriate.
}
finally {
  if (is != null) { is.close(); }
}

http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toByteArray(java.io.InputStream)

Answer 3

Here's a clean solution:

private byte[] downloadUrl(URL toDownload) {
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();

    try {
        byte[] chunk = new byte[4096];
        int bytesRead;
        InputStream stream = toDownload.openStream();

        while ((bytesRead = stream.read(chunk)) > 0) {
            outputStream.write(chunk, 0, bytesRead);
        }

    } catch (IOException e) {
        e.printStackTrace();
        return null;
    }

    return outputStream.toByteArray();
}

Answer 4

I am very surprised that nobody here has mentioned the problem of connection and read timeout. It could happen (especially on Android and/or with some crappy network connectivity) that the request will hang and wait forever.

The following code (which also uses Apache IO Commons) takes this into account, and waits max. 5 seconds until it fails:

public static byte[] downloadFile(URL url)
{
    try {
        URLConnection conn = url.openConnection();
        conn.setConnectTimeout(5000);
        conn.setReadTimeout(5000);
        conn.connect(); 

        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        IOUtils.copy(conn.getInputStream(), baos);

        return baos.toByteArray();
    }
    catch (IOException e)
    {
        // Log error and return null, some default or throw a runtime exception
    }
}

Answer 5

Use commons-io IOUtils.toByteArray(URL):

String url = "http://localhost:8080/images/anImage.jpg";
byte[] fileContent = IOUtils.toByteArray(new URL(url));

Maven dependency:

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.6</version>
</dependency>

Answer 6

byte[] b = IOUtils.toByteArray((new URL( )).openStream()); //idiom

Note however, that stream is not closed in the above example.

if you want a (76-character) chunk (using commons codec)...

byte[] b = Base64.encodeBase64(IOUtils.toByteArray((new URL( )).openStream()), true);

Answer 7

The content length is just a HTTP header. You cannot trust it. Just read everything you can from the stream.

Available is definitely wrong. It's just the number of bytes that can be read without blocking.

Another issue is your resource handling. Closing the stream has to happen in any case. try/catch/finally will do that.

Answer 8

It's important to specify timeouts, especially when the server takes to respond. With pure Java, without using any dependency:

public static byte[] copyURLToByteArray(final String urlStr,
        final int connectionTimeout, final int readTimeout) 
                throws IOException {
    final URL url = new URL(urlStr);
    final URLConnection connection = url.openConnection();
    connection.setConnectTimeout(connectionTimeout);
    connection.setReadTimeout(readTimeout);
    try (InputStream input = connection.getInputStream();
            ByteArrayOutputStream output = new ByteArrayOutputStream()) {
        final byte[] buffer = new byte[8192];
        for (int count; (count = input.read(buffer)) > 0;) {
            output.write(buffer, 0, count);
        }
        return output.toByteArray();
    }
}

Using dependencies, e.g., HC Fluent:

public byte[] copyURLToByteArray(final String urlStr,
        final int connectionTimeout, final int readTimeout)
                throws IOException {
    return Request.Get(urlStr)
            .connectTimeout(connectionTimeout)
            .socketTimeout(readTimeout)
            .execute()
            .returnContent()
            .asBytes();
}