0

I need to encrypt some message before sending it as request body in an API. I have to do this in Python and the library I used is Pycryptodome. From hereon, when I use the term API, I am referring to the API endpoint provided by the vendor and not the Pycryptodome API. The receiver(API vendor) shared the certificate (RSA 2048 bits) which I need to use to encrypt this message. This certificate is in binary format (DER) and has .cer as the extension. I need to import this using Pycryptodome and from the docs (https://pycryptodome.readthedocs.io/en/latest/src/public_key/rsa.html), I find that the import_key method accepts X.509 certificates in either binary or PEM encoding. The API documentation says I need to use RSA/NONE/OAEPWithSHA1AndMGF1Padding for encryption. This is my code.

def encrypt_message(message):
        with open('/home/krishna/work/.certificates/random_certificate.cer', 'rb') as _file:
            key_contents = _file.read()
        recipient_key = RSA.import_key(key_contents)
        cipher_rsa = PKCS1_OAEP.new(recipient_key)
        return base64.b64encode(cipher_rsa.encrypt(message.encode())).decode()

I read the PKCS1_OAEP.py file, the default mask generation function is MGF1, default hash algorithm is SHA1. Hence, I did not specify these parameters in the new function in the above code. Also, I need to send this encrypted message as a base64 string (UTF-8). After reading the API documentation and the Pycryptodome documentation, I felt this code is sufficient for encryption. Also, this code is somewhat standard and is almost the same as the one in the Pycryptodome documentation. However, I get a decryption error and the vendor tells me this is due to "BAD PADDING EXCEPTION". This should not happen since I used OAEP for padding and the vendor confirmed they are using OAEP for padding as well. This is very hard for me to debug as I do not possess the vendor's private key.

So I tried generating my own pair of private and public keys (RSA-2048) and checked if I am able to decrypt my message. And duh, I am able to decrypt my message. Here is the code for encryption and decryption using my own keys.

def encrypt_message(message):
        with open('my_pub.pem', 'rb') as _file:
            pub = _file.read()
        recipient_key = RSA.importKey(pub)
        cipher_rsa = PKCS1_OAEP.new(recipient_key)
        return base64.b64encode(cipher_rsa.encrypt(message.encode())).decode()

def decrypt_message(gibberish):
        with open('my_priv.pem', 'rb') as _file:
            priv =  _file.read()
        pvt_key = RSA.importKey(priv)
        cipher_rsa = PKCS1_OAEP.new(pvt_key)
        return cipher_rsa.decrypt(base64.b64decode(gibberish.encode())).decode()

So where I am going wrong? They key difference between this test and the actual API call is that my keys are generated in PEM format and have .pem as the extension. In the case of the API call, I have to use a .cer certificate in DER format. But I am assuming Pycryptodome is handling this for me (from the docs).

I have a few more questions regarding this.

  1. What is NONE in RSA/NONE/OAEPWithSHA1AndMGF1Padding?
  2. What is the difference between import_key() and importKey() in Pycryptodome? There are 2 methods and it does not look like importKey() has some documentation to it.
  3. Can padding exceptions occur while using OAEP? From what I read from the internet, OAEP is the best method out there, not only in terms of adding randomness during encryption, but I also read that padding exceptions are very rare when this is used.

I really need to know what is going wrong here as I am dead clueless. Any suggestions or help will be appreciated.

  • Is `RSA/NONE/OAEPWithSHA1AndMGF1Padding` really all the information you can provide about the endpoint? In Java/BC that specifies RSA/OAEP with SHA-1 for both digests, so would correspond to the Python/PyCryptodome implementation and would therefore not be an explanation for the issue. Without further information (code, link to documentation, test data etc.) probably only guesses can be made. – Topaco Feb 18 '23 at 16:43
  • Re. the questions: 1. The middle part has no meaning for RSA, therefore `NONE` (for symmetric ciphers the mode of operation is specified). 2. There is none, `importKey()` is for backward compatibility. 3. with the same padding (and padding parameters) on both sides, correct key and ciphertext no exception should occur. – Topaco Feb 18 '23 at 17:00
  • @Topaco "In Java/BC that specifies RSA/OAEP with SHA-1 for both digests, so would correspond to the Python/PyCryptodome implementation and would therefore not be an explanation for the issue." Can you elaborate on this a little more? What do you mean "for both digests"? – Krishna Palle Feb 19 '23 at 08:35
  • Have a look at the description of OAEP in [RFC8017](https://www.rfc-editor.org/rfc/rfc8017#section-7.1.1): OAEP has various parameters, a mask generation function, a label, a content digest used to hash the label, and a digest used by the mask generation function. The default values from RFC8017 are: MGF1 for the mask generation function, an empty label, and SHA-1 for both digests. – Topaco Feb 19 '23 at 09:40
  • `RSA/NONE/OAEPWithSHA1AndMGF1Padding` (if it refers to Java/BC, which you still haven't confirmed!) and the PyCryptodome code use *exactly* these values (the former results from a comparison with the *explicitly* specified parameters, *analogous* as [here](https://stackoverflow.com/a/32166210/9014097), the latter from the docs (s. [`PKCS1_OAEP.new()`](https://pycryptodome.readthedocs.io/en/latest/src/cipher/oaep.html?highlight=oaep#Crypto.Cipher.PKCS1_OAEP.new)). So, since padding and padding parameters are identical, they cannot be the cause of the failure. – Topaco Feb 19 '23 at 09:44
  • @Topaco The API documentation uses Java/BC for reference. I thought `RSA/NONE/OAEPWithSHA1AndMGF1Padding` is a general representation but maybe it is just a BC thing. The vendor confirmed the hash algorithm is SHA-1 and the object returned from PKCS1_OAEP.new() uses this by default, hence I didnt feel the need to explicitly specify. Everything in this representation `RSA/NONE/OAEPWithSHA1AndMGF1Padding` is being used from my knowledge. – Krishna Palle Feb 19 '23 at 10:01
  • @Topaco The link you put up here https://www.rfc-editor.org/rfc/rfc8017#section-7.1.1 says there is only one hash algorithm as input. What is the second algo you are talking about? Is it that MGF uses 1 hash algorithm and OAEP uses another? – Krishna Palle Feb 19 '23 at 10:02
  • 1
    Right, as already said: *...a content digest used to hash the label, and a digest used by the mask generation function...* The content digest is the hash from [7.1.1](https://www.rfc-editor.org/rfc/rfc8017#section-7.1.1) (for hashing the label). The other digest is the MGF1 digest ([B.2.1](https://www.rfc-editor.org/rfc/rfc8017#appendix-B.2.1)). The default values are MGF1 and SHA-1 for both digests ([A.2.1](https://www.rfc-editor.org/rfc/rfc8017#appendix-A.2.1)). – Topaco Feb 19 '23 at 10:34
  • @KrishnaPalle Only BC specifies "RSA/NONE/OAEPWithSHA1AndMGF1Padding". The "NONE" is only picked up by Bouncy Castle though, as JCA always (incorrectly) uses ECB as mode of operation. Replace NONE with ECB and the default providers would also work. Padding exceptions for OAEP are relatively rare yes, usually the same hash is used for the *empty* label. But the main time you get padding errors is when using **the wrong key**, i.e. the public and private key are part of different key pairs. Ask the other party for all the parameters of the OAEP structure and the fingerprint of the certificate. – Maarten Bodewes Feb 19 '23 at 15:09
  • Just tested it and using `"OAEPWithSHA1AndMGF1Padding"` indeed is the same as using `new OAEPParameterSpec("SHA1", "MGF1", new MGF1ParameterSpec("SHA1"), PSource.PSpecified.DEFAULT)` for `"OAEPPadding"`, i.e. just SHA-1 for everything and an empty label. No differences between Bouncy Castle or the default provider seen (on Java 17, BC 1.72). – Maarten Bodewes Feb 19 '23 at 15:19
  • You can also simply test the compatibility with a certificate and an associated private key: [Here](https://replit.com/@3hK8cL8H24hwiS7/WorstOverdueStructs#main.py) the Python code you posted can be run online and [here](https://replit.com/@3hK8cL8H24hwiS7/GreedyTrickyMachinecodeinstruction#Main.java) the Java/BC code. – Topaco Feb 19 '23 at 18:35
  • Result: The ciphertext generated with the Python/PyCryptodome code can be decrypted with the Java/BC code (as expected). So your problem is obviously somewhere else, possibly a key issue, caused by corrupted data, a misunderstanding about padding, etc. – Topaco Feb 19 '23 at 19:01
  • @MaartenBodewes Even I felt this could be happening because I was encrypting with the wrong public key but I did not get a clear response regarding this from the vendor so I first want to ensure what I am doing is right. – Krishna Palle Feb 19 '23 at 19:39
  • @MaartenBodewes What is the "default provider" you mention in the comment? – Krishna Palle Feb 19 '23 at 19:42
  • @Topaco This test is good. I see that the certificate you stored under `pub` in the python side is a utf-8 string in PEM format. What happens if I use a certificate in DER format and store it in `pub` as bytes? – Krishna Palle Feb 19 '23 at 20:01
  • Works as well since [`import_key()`](https://pycryptodome.readthedocs.io/en/latest/src/public_key/rsa.html?highlight=import_key#Crypto.PublicKey.RSA.import_key) can handle both PEM and DER encoded certificates/keys, run the code online [here](https://replit.com/@3hK8cL8H24hwiS7/LuminousJumboPup#main.py). – Topaco Feb 19 '23 at 20:32
  • @KrishnaPalle SunJCE version 17 (for Java 17, makes sense I suppose :) ) The providers implement the algorithms (e.g. `CipherSpi`), while `Cipher` is simply an object that wraps the various implementations; this allows providers to "plug in" their own algorithm implementations. – Maarten Bodewes Feb 19 '23 at 21:37
  • @Topaco So I guess the problem narrows down to either the OAEP parameters or key mismatch. What do you think? – Krishna Palle Feb 20 '23 at 04:18
  • @KrishnaPalle - Contact the vendor and make sure that the padding information is correct and that the certificate is the correct one (i.e. is associated with the private key). It would be helpful if the vendor could provide the decryption code. Regarding the padding, you can proactively try other OAEP parameters on the off chance, e.g. both digests as SHA-256 or SHA-1/SHA-256 combinations, or PKCS#1 v1.5 Padding. – Topaco Feb 20 '23 at 08:28

0 Answers0