Encryption/Decryption with bouncycastle-java and RSAES-OAEP

Question

I am not an expert with encryption, but i am trying to create an CMSEnvelopedDataGenerator with bouncycastle 1.67, where the session key is encrypted with RSAES-OAEP (1.2.840.113549.1.1.7)

For now my code looks like this:

 CMSEnvelopedDataGenerator envelopedGenerator = new CMSEnvelopedDataGenerator();
 
 JcaAlgorithmParametersConverter paramsConverter = new JcaAlgorithmParametersConverter();
 OAEPParameterSpec oaepSpec = new OAEPParameterSpec("SHA-256", "MGF1", MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);

 AlgorithmIdentifier algorithmIdentifier;    
 algorithmIdentifier = paramsConverter.getAlgorithmIdentifier(PKCSObjectIdentifiers.id_RSAES_OAEP, oaepSpec);
 JceKeyTransRecipientInfoGenerator recipent = new JceKeyTransRecipientInfoGenerator(receiverCert, algorithmIdentifier).setProvider("BC");    
 
 # encrypt
 CMSEnvelopedData envelopedData;
 envelopedData = envelopedGenerator.generate(
     new CMSProcessableByteArray(encodedSignedData),  
     new JceCMSContentEncryptorBuilder(CMSAlgorithm.AES256_CBC).setProvider("BC").build()
 )

It runs through but when i check it via openssl asn1parse, i see

115:d=6  hl=2 l=   9 prim: OBJECT            :rsaesOaep
126:d=6  hl=2 l=  47 cons: SEQUENCE
128:d=7  hl=2 l=  15 cons: cont [ 0 ]
130:d=8  hl=2 l=  13 cons: SEQUENCE
132:d=9  hl=2 l=   9 prim: OBJECT            :sha256
143:d=9  hl=2 l=   0 prim: NULL
145:d=7  hl=2 l=  28 cons: cont [ 1 ]
147:d=8  hl=2 l=  26 cons: SEQUENCE
149:d=9  hl=2 l=   9 prim: OBJECT            :mgf1
160:d=9  hl=2 l=  13 cons: SEQUENCE
162:d=10 hl=2 l=   9 prim: OBJECT            :sha256

and then the hex dump. On my reference file it is like:

115:d=6  hl=2 l=   9 prim: OBJECT            :rsaesOaep
126:d=6  hl=2 l=  43 cons: SEQUENCE
128:d=7  hl=2 l=  13 cons: cont [ 0 ]
130:d=8  hl=2 l=  11 cons: SEQUENCE
132:d=9  hl=2 l=   9 prim: OBJECT            :sha256
143:d=7  hl=2 l=  26 cons: cont [ 1 ]
145:d=8  hl=2 l=  24 cons: SEQUENCE
147:d=9  hl=2 l=   9 prim: OBJECT            :mgf1
158:d=9  hl=2 l=  11 cons: SEQUENCE
160:d=10 hl=2 l=   9 prim: OBJECT            :sha256

On line 143 at my file is the line

143:d=9  hl=2 l=   0 prim: NULL

I am not sure where that comes from.

When i use my decryption code, which works for my reference file, i am getting the following exceptions

 exception unwrapping key: bad padding: unable to decrypt block

Caused by: org.bouncycastle.cms.CMSException: exception unwrapping key: bad padding: unable to decrypt block
 at org.bouncycastle.cms.jcajce.JceKeyTransRecipient.extractSecretKey(Unknown Source)
 at org.bouncycastle.cms.jcajce.JceKeyTransEnvelopedRecipient.getRecipientOperator(Unknown Source)
 at org.bouncycastle.cms.KeyTransRecipientInformation.getRecipientOperator(Unknown Source)
 at org.bouncycastle.cms.RecipientInformation.getContentStream(Unknown Source)

Caused by: org.bouncycastle.operator.OperatorException: bad padding: unable to decrypt block
at org.bouncycastle.operator.jcajce.JceAsymmetricKeyUnwrapper.generateUnwrappedKey(Unknown Source)

Caused by: org.bouncycastle.jcajce.provider.util.BadBlockException: unable to decrypt block
at org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.getOutput(Unknown Source)
at org.bouncycastle.jcajce.provider.asymmetric.rsa.CipherSpi.engineDoFinal(Unknown Source)
at javax.crypto.Cipher.doFinal(Cipher.java:2168)

Caused by: org.bouncycastle.crypto.InvalidCipherTextException: data wrong
at org.bouncycastle.crypto.encodings.OAEPEncoding.decodeBlock(Unknown Source)
at org.bouncycastle.crypto.encodings.OAEPEncoding.processBlock(Unknown Source)

I hope its not much, that is missing.

Edit:

With my wrong generated file recipient.getKeyEncryptionAlgorithm().getParameters() results in

[[0][2.16.840.1.101.3.4.2.1, NULL], [1][1.2.840.113549.1.1.8, [2.16.840.1.101.3.4.2.1, NULL]]]

the correct file in

[[0][2.16.840.1.101.3.4.2.1], [1][1.2.840.113549.1.1.8, [2.16.840.1.101.3.4.2.1]]]

From where are these wrong NULL for SHA-256 values coming.

Answer 1

You only mention one 'extra' NULL in the BC-created message, but actually there are two, with the second one on the first line you excluded from the data you posted. The (different) length fields in your post, as well as the display of getParameters(), clearly show this.

Those NULLs are not wrong.

Those NULLs are the parameters for the hash algorithms in the parameters structure for OAEP, and are required by the standard. From RFC 3447 = PKCS1v2.1 which is the first version to include SHA-2 (in 2003, just after FIPS 186-2 in 2002) at A.2.1:

The parameters field ... shall have a value of type RSAES-OAEP-params:

      RSAES-OAEP-params ::= SEQUENCE {
          hashAlgorithm     [0] HashAlgorithm    DEFAULT sha1,
          maskGenAlgorithm  [1] MaskGenAlgorithm DEFAULT mgf1SHA1,
          pSourceAlgorithm  [2] PSourceAlgorithm DEFAULT pSpecifiedEmpty
      }
...
         HashAlgorithm ::= AlgorithmIdentifier {
            {OAEP-PSSDigestAlgorithms}
         }

         OAEP-PSSDigestAlgorithms    ALGORITHM-IDENTIFIER ::= {
             { OID id-sha1 PARAMETERS NULL   }|
             { OID id-sha256 PARAMETERS NULL }|
             { OID id-sha384 PARAMETERS NULL }|
             { OID id-sha512 PARAMETERS NULL },
             ...  -- Allows for future expansion --
         }
...
         MaskGenAlgorithm ::= AlgorithmIdentifier {
            {PKCS1MGFAlgorithms}
         }
         PKCS1MGFAlgorithms    ALGORITHM-IDENTIFIER ::= {
             { OID id-mgf1 PARAMETERS HashAlgorithm },
             ...  -- Allows for future expansion --
         }

Observe that both hash specifications -- the outer hash for the label and the inner hash within MGF1 parameters -- are defined by infoset HashAlgorithm and all the defined values in that set, including SHA-256, have parameters explicitly NULL, not omitted as the generic X.509 ASN.1 permits (compare RFC5280 4.1.1.2 which uses the older pre-infoset notation).

Note the same is true for PSS in A.2.3, and with a slightly larger set of hash algorithms for the DigestInfo within RSASSA-PKCS1-v1_5 in A.2.4. This, and equivalent provisions in v2.0 (excluding PSS which wasn't in 2.0, and with slightly different notation), may be in reaction to PKCS1v1.5 at 10.1.2 only making signature DigestInfo parameters a 'should' (lowercase even after 2119 presumably because this was RSALabs text not IETF), which led to variation in implementations causing signatures which were in fact correct sometimes to not verify, which was considered to be a bad thing in need of fixing.

Thus your 'reference' file is technically in violation of the standard. However, since these hash algorithms don't actually use parameters -- that's why they are encoded with NULL -- BouncyCastle can easily be tolerant and accept the omitted case. I tested with an otherwise valid structure and it does work both ways. (It wouldn't surprise me if it even works with some inappropriate value like an octet string inserted there, but I didn't test that.)

Even if the parameters encoding were wrong, it wouldn't cause the exception you have -- it would either be an explicit decoding/parsing error something like 'required field missing' or an instantiation error something like 'invalid parameters for algorithm x'. In the absence of bugs, 'bad padding' is caused by damaged, tampered or otherwise wrong data (which in a CMS environment is very unlikely) or mismatched keys.

Check that you are using matching keys.