Java InputStream: copy to and calculate hash at the same time

Question

Here are my two code snippets:

public class Uploader {

  private static final String SHA_256 = "SHA-256";

  public String getFileSHA2Checksum(InputStream fis) throws IOException {
    try {
      MessageDigest md5Digest = MessageDigest.getInstance(SHA_256);
      return getFileChecksum(md5Digest, fis);
    } catch (NoSuchAlgorithmException e) {
      return "KO";
    }
  }

  public void transferTo(InputStream fis) throws IOException {
    FileUtils.copyInputStreamToFile(fis, file2);
  }

My code uses this class as:

Is it possible to copyToFile and calculateChecksum at the same time leveraging InputStream is open?

What do you mean by "at the same time"? Concurrency? Or do you just want to read the input stream once? — Sweeper, Apr 14 '23 at 09:26
Some kind of chaining... I need to store inputstream to a file, but I'd like to calculate checksum since inputstream is being read... At the end, I need checksum and a file stored. Yes I want to read input stream once. — Jordi, Apr 14 '23 at 09:30

Mark Rotteveel · Answer 1 · 2023-04-14T11:18:20.090

You can use the DigestInputStream to calculate a hash while reading from a stream. That is, you wrap the original input stream with a DigestInputStream and read through the DigestInputStream. While reading the data, the message digest is automatically updated, and you can retrieve the digest after you read the entire stream.

Alternatively, you can use DigestOutputStream to calculate a hash while writing to a stream. In a similar vein, you wrap the destination output stream with a DigestOutputStream and write through the DigestOutputStream.

A quick and dirty example:

var inputFile = Path.of("D:\\Development\\data\\testdata-csv\\customers-1000.csv");
var outputFile = Files.createTempFile("testoutput", ".dat");
var md = MessageDigest.getInstance("SHA-256");
try (var in = new DigestInputStream(Files.newInputStream(inputFile), md);
     var out = Files.newOutputStream(outputFile)) {
    in.transferTo(out);
} finally {
    Files.deleteIfExists(outputFile);
}

System.out.println(HexFormat.of().formatHex(md.digest()));

In terms of your existing code, you could do something like:

public String transferAndHash(InputStream in) throws IOException {
    try {
        var md = MessageDigest.getInstance("SHA-256");
        try (var digestIn = new DigestInputStream(in, md)) {
            transferTo(digestIn);
        }
        return HexFormat.of().formatHex(md.digest());
    } catch (NoSuchAlgorithmException e) {
        // all recent Java versions are required to support SHA-256
        throw new AssertionError("Expected SHA-256 to be supported", e);
    }
}

(NOTE: HexFormat was introduced in Java 17, if you're using an earlier version, you'll need an alternative.)

score 2 · Accepted Answer · answered Apr 14 '23 at 10:25

2

Here is an example using DigestInputStream based on Mark Rotteveel's suggestion (which I liked):

Path input = Paths.get("input_file");
Path output = Paths.get("output_file");
MessageDigest algorithm = MessageDigest.getInstance("SHA-256");
try (InputStream is = Files.newInputStream(input);
     DigestInputStream hashingStream = new DigestInputStream(is, algorithm)) {
    Files.copy(hashingStream, output);
}
byte[] digest = algorithm.digest();
// this line uses Apache Commons Codec to show the hex representation of the byte[]
String hash = Hex.encodeHexString(digest);

answered Apr 14 '23 at 10:25

Miss Chanandler Bong

4,081
10
26
36

Java 17 comes with [`HexFormat`](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/HexFormat.html), so you don't need to use external libraries anymore. – Mark Rotteveel Apr 14 '23 at 10:47
@MarkRotteveel I didn’t use jdk 17 yet. Thanks for the suggestion. – Miss Chanandler Bong Apr 14 '23 at 10:55
My input stream is an http form upload multipart stream... I need to delete file if user cancels upload... Any ideas? – Jordi Apr 14 '23 at 11:46
@Jordi you can add a catch block to the try and use `Files.deleteIfExists` inside it. `try (...) { ... } catch (Exception ex) { Files.deleteIfExists(output); }`. – Miss Chanandler Bong Apr 14 '23 at 12:07

score 1 · Answer 3 · answered Apr 14 '23 at 10:10

If you want to read the input stream once, you might manual read the bytes from the input string and write it to you file by yourself.

    private static final String SHA_256 = "SHA-256";
    private static final int BUFFER = 1024;

    public String writeToFile(InputStream fis, File outputFile) throws IOException {
        try (var fout = new FileOutputStream(outputFile); fis) {
            MessageDigest md5Digest = MessageDigest.getInstance(SHA_256);
            var buffer = new byte[BUFFER];
            var bytesRead = 0;
            while (true) {
                bytesRead = fis.read(buffer);
                if (bytesRead < 0)
                    break;
                md5Digest.update(buffer, 0, bytesRead);
                fout.write(buffer, 0, bytesRead);
            }
            var checksume = md5Digest.digest();
            return Hex.encodeHexString(checksume);
        } catch (NoSuchAlgorithmException e) {
            return "KO";
        }
    }

Java InputStream: copy to and calculate hash at the same time

3 Answers3