Intermittent IllegalBlockSizeException when doing RSA decryption using KeyStore

Question

The RSA key is generated using the following code:

RSAKeyGenParameterSpec rsaSpec = new RSAKeyGenParameterSpec(2048, RSAKeyGenParameterSpec.F4);
AlgorithmParameterSpec keyPairGeneratorSpec;

KeyGenParameterSpec.Builder specBuilder = new KeyGenParameterSpec.Builder(keyAlias, KeyProperties.PURPOSE_DECRYPT)
  .setAlgorithmParameterSpec(rsaSpec)
  .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_RSA_OAEP, KeyProperties.ENCRYPTION_PADDING_RSA_PKCS1)
  .setDigests(KeyProperties.DIGEST_SHA256);
keyPairGeneratorSpec = specBuilder.build();

try {
  KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance(RSA, ANDROID_KEYSTORE);
  keyPairGenerator.initialize(keyPairGeneratorSpec);
  keyPairGenerator.generateKeyPair();
} catch(Exception e) {
  //handle exception
}

And then it is used to encrypt remotely on a Java server using:

Security.addProvider(new BouncyCastleProvider());
Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding", "BC");
cipher.init(Cipher.ENCRYPT_MODE, publicKey);
cipher.doFinal(data);

And then on the device, the decryption is done like this:

Cipher rsaCipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
rsaCipher.init(Cipher.DECRYPT_MODE, key);
return rsaCipher.doFinal(data);

A small percentage of the time, the decryption occurs successfully, but most of the time I get this exception:

javax.crypto.IllegalBlockSizeException
  at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineDoFinal(AndroidKeyStoreCipherSpiBase.java:519)
  ...
Caused by: android.security.KeyStoreException: Unknown error
  at android.security.KeyStore.getKeyStoreException(KeyStore.java:695)
  at android.security.keystore.KeyStoreCryptoOperationChunkedStreamer.doFinal(KeyStoreCryptoOperationChunkedStreamer.java:224)
  at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineDoFinal(AndroidKeyStoreCipherSpiBase.java:506)
  at javax.crypto.Cipher.doFinal(Cipher.java:1736)
  ...

I found this exception in a couple other SO answers, including this one: https://stackoverflow.com/a/36020975/6552833 but it doesn't seem relevant since I am not using OAEP encryption in the first place.

You should use OAEP (Optimal Asymmetric Encryption Scheme). See Maarten's great [answer](https://crypto.stackexchange.com/a/26539/18298) or Thomas's [answer](https://stackoverflow.com/questions/2855326/how-can-i-create-a-key-using-rsa-ecb-pkcs1padding-in-python/2856628#2856628) — kelalaka, Feb 24 '19 at 18:48
The code shown is not likely the source of the error. The error is most likely due to the code used to encode the encryption into a payload to be sent to the peer and/or the decoding of the received data prior to attempting decryption. I would bet you are treating the output of encryption incorrectly as a string somewhere. — President James K. Polk, Feb 24 '19 at 20:11

score 2 · Answer 1 · edited Sep 18 '20 at 01:21

I had the same issue. In my case I found 2 causes:

Server side encryption was somehow adding an extra byte so the encrypted base64 string had 257 bytes instead of 256 which is what the cipher expects. Stripping the first byte made it work on most devices.
Manifest file setting - android: allowsbackup ="true" was causing the decryption to fail on certain devices (release build. Debug builds were fine). Setting this to False fixed it.

<application
android:name=".application.MyApplication"
tools:replace="android:allowBackup"
android:allowBackup="false">

Intermittent IllegalBlockSizeException when doing RSA decryption using KeyStore

1 Answers1