In case AES/ECB/PKCS5Padding is used you get a BadPaddingException when trying to decrypt a message with the wrong key. But there is no exception when AES/ECB/NoPadding is used - message is successfully decrypted to some wrong data.
Is there a way to detect the wrong cipher key during AES decryption when no padding is used?
Edit:
I'm communicating with the device which has encryption set to AES/ECB/NoPadding. I think I can't change this.
PKCS#5 padding will successfully decrypt with a probability of a little more than 2-8. Every 256th random key that you try will give you a wrong plaintext without throwing an exception1. Checking padding is necessary, but it is not a good way to determine whether the key is correct.
The same numbers are true for ANSI X.923 padding, but ISO 10126 padding is different. Padding is at most a full block and at least a single byte, so valid values for ISO10126Padding are 0x01 through 0x10. Since it is randomized only the last padding byte is used and therefore roughly 16/256 ~ 6% wrong keys won't throw an exception.
A way check the validity of the key before trying to decrypt the full ciphertext is by sending a Key Check Value (KCV) along with the ciphertext.
This is simply an encryption of a full zero-filled block with the key. Since block ciphers like AES are (currently) not vulnerable to key recovery through known-plaintext attacks, this should give a good way to check for a valid key.
You could be using HMAC to check for key integrity. Pseudo-code:
byte[] salt = generateRandomByteArray(16);
byte[] keyTag = hmacSha256(key, salt);
return keyTag + salt + encrypt(data, key);
At the receiver side you can read the salt, run the key and the salt through HMAC and compare with the received keyTag. But that only checks whether the key is correct. You really should be checking whether the whole combination of key and ciphertext is correct.
This property is called authenticated encryption and it can be achieved by running the keyed HMAC over the ciphertext to create the authentication tag. You would then need to generate two keys through HKDF (pseudo-code):
byte[] keyEnc = hkdf(key, "Encryption");
byte[] keyMac = hkdf(key, "MAC");
byte[] ciphertext = encrypt(data, keyEnc);
byte[] authTag = hmacSha256(keyMac, ciphertext);
return ciphertext + authTag;
In its most basic form, HKDF is simply a double invocation of HMAC. It might be easier to just use an authenticated mode like GCM or EAX.
If you cannot add anything to your current encryption procedure, then there are still probabilistic ways to determine whether the key was correct. The main idea is that the recovered plaintext has to be examined whether it makes sense.
For example, if your real plaintext is some text then you can check whether the decrypted plaintext contains only characters that would be available in ASCII or if you encrypted UTF-8, you can check whether it actually properly decodes it. If the original plaintext was some JSON, then you can run a JSON validator to check that you get valid JSON and be relatively sure that the correct key was applied.
Keep in mind that an attacker might still tamper with the ciphertext to produce a related plaintext that properly verifies. The success rate heavily depends on the structure of your plaintext.
1 When you're having random keys, then the chance that the last bytes of the decrypted plaintext before unpadding is a 0x01 byte is 1 in 256 which is a valid padding regardless what comes before it. You have a 1 in 256*256 chance to get a 0x0202 in the last two bytes which is also a valid padding.
Please don't use ECB mode. It's not particularly secure since it doesn't provide semantic security. You should at least be using CBC mode with a randomly generated IV.
Implementations: HMAC-SHA256, AES-GCM
References: HKDF
