0

I have working code that uses Java Bouncycastle lib that (as I understood) decrypts CMS data.

byte[] encryptedData = Base64.decode(encryptedText);
CMSEnvelopedData cmsEnvelopedData = new CMSEnvelopedData(encryptedData);
RecipientInformationStore recipients = cmsEnvelopedData.getRecipientInfos();
X509CertificateHolder decryptPublicCertificateHolder = new X509CertificateHolder(decryptPublicCertificate.getEncoded());

for (RecipientInformation recipient : recipients) {
    org.bouncycastle.cms.RecipientId recipientId = recipient.getRID();          
    Recipient decryptRecipient = new JceKeyTransEnvelopedRecipient(decryptPrivateKey);
    byte[] plainData = recipient.getContent(decryptRecipient);
    String plainText = new String(plainData, getEncoding());
    // ...      
}
// ...

Certificate and private key are stored together in .pfx file. Encrypted message comes as string in Base64

To decrypt the message using openssl I:

  1. Extracted cert and private key into 2 separate PEM files:
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

Private key is decrypted

-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
  1. Added header/footer to encrypted message
-----BEGIN PKCS7-----
...
-----END PKCS7-----
  1. Tried to decrypt the message
openssl cms -decrypt -inform PEM -in encrypted_message_file -inkey private_key.pem -recip certificate.pem

As a result I get

Error decrypting CMS structure
16972:error:060CC07A:digital envelope routines:EVP_CIPHER_asn1_to_param:cipher parameter error:../openssl-1.1.1e/crypto/evp/evp_lib.c:79:
16972:error:2E078066:CMS routines:cms_EncryptedContent_init_bio:cipher parameter initialisation error:../openssl-1.1.1e/crypto/cms/cms_enc.c:80

I also tried remove header/footer for encrypted message, tried provide .pfx file as private key and many more but no result

I'm very new to that stuff. Could you please suggest what am I doing wrong?

UPD1: openssl pkcs7 -in encrypted_message_file -text produces

unable to load PKCS7 object
2496:error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag:../openssl-1.1.1e/crypto/asn1/tasn_dec.c:1130:
2496:error:0D07803A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error:../openssl-1.1.1e/crypto/asn1/tasn_dec.c:290:Type=PKCS7_ISSUER_AND_SERIAL
2496:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:../openssl-1.1.1e/crypto/asn1/tasn_dec.c:627:Field=issuer_and_serial, Type=PKCS7_RECIP_INFO
2496:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:../openssl-1.1.1e/crypto/asn1/tasn_dec.c:595:Field=recipientinfo, Type=PKCS7_ENVELOPE
2496:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:../openssl-1.1.1e/crypto/asn1/tasn_dec.c:627:       
2496:error:0D08403A:asn1 encoding routines:asn1_template_ex_d2i:nested asn1 error:../openssl-1.1.1e/crypto/asn1/tasn_dec.c:477:Field=d.enveloped, Type=PKCS7
2496:error:0906700D:PEM routines:PEM_ASN1_read_bio:ASN1 lib:../openssl-1.1.1e/crypto/pem/pem_oth.c:33:

UPD2 in response to dave_thompson_085 comment

openssl asn1parse -i -in encrypted.pem

    0:d=0  hl=4 l=2307 cons: SEQUENCE          
    4:d=1  hl=2 l=   9 prim:  OBJECT            :pkcs7-envelopedData
   15:d=1  hl=4 l=2292 cons:  cont [ 0 ]        
   19:d=2  hl=4 l=2288 cons:   SEQUENCE          
   23:d=3  hl=2 l=   1 prim:    INTEGER           :02
   26:d=3  hl=4 l= 304 cons:    SET               
   30:d=4  hl=4 l= 300 cons:     SEQUENCE          
   34:d=5  hl=2 l=   1 prim:      INTEGER           :02
   37:d=5  hl=2 l=  20 prim:      cont [ 0 ]        
   59:d=5  hl=2 l=  13 cons:      SEQUENCE          
   61:d=6  hl=2 l=   9 prim:       OBJECT            :rsaEncryption
   72:d=6  hl=2 l=   0 prim:       NULL              
   74:d=5  hl=4 l= 256 prim:      OCTET STRING      [HEX DUMP]:512_LENGTH_HEX_STRING
  334:d=3  hl=4 l=1973 cons:    SEQUENCE          
  338:d=4  hl=2 l=   9 prim:     OBJECT            :pkcs7-data
  349:d=4  hl=2 l=  12 cons:     SEQUENCE          
  351:d=5  hl=2 l=   8 prim:      OBJECT            :rc4
  361:d=5  hl=2 l=   0 prim:      NULL              
  363:d=4  hl=4 l=1944 prim:     cont [ 0 ]        
  • In spite of its name `openssl pkcs7` only handles a _subset_ of PKCS7/CMS, namely an empty SignedData used to carry cert(s) and/or CRL(s), sometimes called p7b or p7c. So ignore that. It appears your message is using an encryption type OpenSSL doesn't support; do (and if the results aren't clear, post) `openssl asn1parse -i -in encrypted.pem` – dave_thompson_085 May 10 '22 at 22:42
  • Hello @dave_thompson_085, thanks , I updated my question with the command result. Could you take a look and, probably, clarify based on the result, what is the next steps? – Kanstantsin T. May 10 '22 at 22:56
  • RC4?? Wow, that's a - cough - interesting choice. And it is indeed not implemented in OpenSSL; the EVP_CIPHER instance isn't set up for use in an ASN.1 context like CMS. AFAICS it's the only one standardly in EVP (which includes dubious-er RC2 and singleDES) that isn't. Which isn't much consolation :-( – dave_thompson_085 May 11 '22 at 07:17
  • Not my choice, locked by 3rd party system that sends messages :(. Actually I need to decrypt it via Node.js but there is no Bouncycastle implementation and other libs couldn't process the message. So I almost gave up and started thinking about direct openssl execution from Nodejs. – Kanstantsin T. May 11 '22 at 07:50

1 Answers1

0

Okay. My initial goal was to decrypt CMS message. In my case the message contains data key encrypted by rsa and content encrypted by rc-4.

I couldn't decrypt the message using Openssl, instead I used https://github.com/lapo-luchini/asn1js npm package to build ASN1 object from the CMS message, extract encrypted data key, decrypt that data key and decrypt content using that key.

I used https://datatracker.ietf.org/doc/html/rfc5652 to read about CMS structure. Also I used UI to visualize parsed message (https://lapo.it/asn1js/).

UPD1.

  • Eventually we decided to abandon CMS rc4. So my question is irrelevant.
  • No, I didn't find the way to decrypt the message via openssl (probably it's not possible since rc4 is not supported)
  • I decrypt it using nodejs but it was so much pain
  • Is this really an answer to the question? It seems to be more of an update to the question, describing the steps you took. That can be helpful, but I think it would be more helpful if you explained where these steps left you and where you still need help. – erickson May 11 '22 at 16:03
  • OpenSSL doesn't _directly_ support RC4 in CMS/PKCS7/SMIME. It does support the raw cipher, and you _could_ do the same as you coded in js: find the two important bits of ASN.1 (RSA-encrypted DEK and RC4-encrypted data) with `asn1parse` then extract each with `asn1parse -strparse $offset -out $file -noout` and fix the implicit tag on the latter with something like `dd`, then RSA-decrypt the DEK with `rsautl` or `pkeyutl` and convert to hex, and finally decrypt the data with `enc -rc4 -d -K $hex` (uppercase K not lowercase). Rather tedious, especially as you no longer need it. – dave_thompson_085 May 18 '22 at 09:01