Java - Hash algorithms - Fastest implementations

Question

I want to know what is the best and fastest implementation of hash algorithms for Java especially MD5 and SHA-2 512 (SHA512) or 256. I want a function to get a string as an argument and return the hash as the result. Thak you.

Edit: This is for getting mapping each URL to a unique hash. Since MD5 is not that reliable in this area, I'm more interested in finding the best & fastest implementation for SHA-2 algorithms. Note that I know even SHA-2 might produce the same hash for some URLs but I can live with that.

Answer 1

First things first: speed is overrated. You should make measures before declaring that a given algorithm is "too slow". Most of the time, hash function speed makes no noticeable difference anyway. If you have qualms about security, then first select a hash function which is secure enough, and then only worry about performance.

Moreover, you want to hash "strings". A Java String is, internally, a chunk from an array of char values which represent Unicode code points (actually, Unicode 16-bit code units which encode the code points using UTF-16). A hash function takes as input a sequence of bits or bytes. So you will have to make a conversion step, e.g. str.getBytes("UTF-8"), to obtain your string as a bunch of bytes. It is likely that the conversion step will have a non-negligible cost when compared to the hashing itself.

Note: beware of URL-encoding ! In a URL, some bytes can be replaced with sequences beginning with a '%' sign; this is meant to support non-printable characters, but it can be used on "standard" characters as well (e.g., replacing 'a' with '%61'). This means that two strings which are distinct (in the String.equals() sense) may actually represent the same URL (as far as URL processing is concerned). Depending on your situation, this may or may not be an issue.

You should first try to use Java's MessageDigest API with the standard (already installed) JCE provider (i.e. you call MessageDigest.getInstance("SHA-256")), and bench the result. Theoretically, the JCE may map the call to an implementation with "native" code (written in C or assembly), which will be faster than what you can get with Java.

That being said...

sphlib is an opensource implementation of many cryptographic hash functions, in C and in Java. The code has been optimized for speed, and, in practice, the Java version turns out to be faster than what the standard JRE from Sun/Oracle offers. Use this link in case the previous link fails (the main host server is sometimes down for maintenance, as seems to be the case right now)(warning: 10 MB download). The archive also contains a report (which was presented at the second SHA-3 candidate conference in 2010) which gives some measured performance figures on several platforms, for SHA-2 and the 14 "second round" candidates for the upcoming SHA-3.

But you really should make in-situation benchmarks. For instance, effects on L1 cache can have a drastic effect on performance, and cannot be accurately predicted by taking the function code and running it in isolation.

Answer 2

Edit: I originally read the question as what's "the fastest hash algorithm" and it has been clarified to be "the fastest implementation of each algorithm". It's a valid question and others have pointed out faster implementations. However unless you're hashing large amounts of data in a short amount of time, it's simply not going to matter very much. I doubt it's usually worth the time and complexity to use something other than what's provided with the standard JCE.

For URL addresses you'd need to be hashing with SHA-256 upward of a million per second on modern hardware to require something faster. I can't imagine most applications needing more than a thousand per second (over 86 million per day), which means the overall CPU time spent hashing would be far less than 1%. So even if you had an infinitely fast hash algorithm you'd only be able to improve overall performance by 1% at best.

Original Answer: Getting both the best and fastest are at odds with each other. The better hashes are generally slower. If you really need speed and security isn't as much of a concern then use MD5. If you need the best security then go with SHA-256 or even SHA-512. You haven't mentioned what you're using it for so it's hard to recommend one or the other. You're probably safest going with SHA-256, as it should be fast enough for most use cases on modern hardware anyhow. Here's how you can do it:

String input = "your string";
MessageDigest digest = MessageDigest.getInstance("SHA-256");
digest.update(input.getBytes("UTF-8"));
byte[] hash = digest.digest();

If you're using this for security purposes, like hashing a password, then you should add salt to the digest as well. If you want a printable string out of the hash, you can encode it back to a string as hex:

static char[] HEX_CHARS = "0123456789ABCDEF".toCharArray();

StringBuilder sb = new StringBuilder(hash.length * 2);
for (byte b : hash) {
    sb.append(HEX_CHARS[(b & 0xF0) >> 4]);
    sb.append(HEX_CHARS[b & 0x0F]);
}
String hex = sb.toString();

Answer 3

Consider BLAKE2 which is faster and more secure than the hashes mentioned above.

MD5, SHA-1, SHA256, and SHA-512 are susceptible to length-extension.

MD5 and SHA-1 are vulnerable to collisions.

MD5 is vulnerable to chosen-prefix collisions.

SHA-3 and BLAKE2 have no known security issues and can generate digests of varying length.

SHA-3 is fastest when implemented in hardware; BLAKE2 is fastest when using software implementations.

BLAKE2b is optimized for 64-bit platforms and produces digests of any size between 1 and 64 bytes.

BLAKE2s is optimized for 8 to 32-bit platforms and produces digests of any size between 1 and 32 bytes.

Here are benchmarks for AES, MD5, SHA-256, and BLAKE2b.

https://blake2.net/

https://www.cryptopp.com/benchmarks.html

In the first link, BLAKE2b (947 Mbits) is much faster than SHA-256 (413 Mbits) and MD5 (632 Mbits).

In the second link, AES-256 CBC (805 Mbits) and BLAKE2b (776 Mbits) are about equal in speed and faster then SHA-256 (275 Mbits) and MD5 (602) Mbits.

Answer 4

Check these out: Lots of SHA / MD5 examples

Also: From same thread: Fast MD5

String hash = MD5.asHex(MD5.getHash(new File(filename)));

Answer 5

Another thing to consider is using MD4. It is not as safe as MD5, but is computed even faster. Windows up to XP used to store and exchange passwords in MD4, so we use this hash because it still allows us to provide authentication services to this platform.

Answer 6

For a string, just call the hashCode() because is cheaper in memory overhead.

Otherwise I recommend this code for private hash:

public static int hash8(String val) throws UnsupportedEncodingException {
    return hash8(val.getBytes("UTF-8"));
}

public static int hash8(byte[] val) {
    int h = 1, i = 0;
    for (; i + 7 < val.length; i += 8) {
        h = 31 * 31 * 31 * 31 * 31 * 31 * 31 * 31 * h + 31 * 31 * 31 * 31
                * 31 * 31 * 31 * val[i] + 31 * 31 * 31 * 31 * 31 * 31
                * val[i + 1] + 31 * 31 * 31 * 31 * 31 * val[i + 2] + 31
                * 31 * 31 * 31 * val[i + 3] + 31 * 31 * 31 * val[i + 4]
                + 31 * 31 * val[i + 5] + 31 * val[i + 6] + val[i + 7];
    }
    for (; i + 3 < val.length; i += 4) {
        h = 31 * 31 * 31 * 31 * h + 31 * 31 * 31 * val[i] + 31 * 31
                * val[i + 1] + 31 * val[i + 2] + val[i + 3];
    }
    for (; i < val.length; i++) {
        h = 31 * h + val[i];
    }
    return h;
}

FYI: http://lemire.me/blog/2015/10/22/faster-hashing-without-effort/