26

Java has a mode called RSA/ECB/OAEPWithSHA-256AndMGF1Padding. What does that even mean?

RFC3447, Public-Key Cryptography Standards (PKCS) #1: RSA Cryptography Specifications Version 2.1, section 7.1.2 Decryption operation says Hash and MGF are both options for RSAES-OAEP-DECRYPT. MGF is it's own function, defined in Section B.2.1 MGF1 and that has it's own Hash "option" as well.

Maybe the Hash "option" in RSAES-OAEP-DECRYPT and MGF1 are supposed to be the same or maybe they're not, it is unclear to me. If they are then I guess when you have RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING that means sha256 should be used for both. But if they're not supposed to be the same then you could have sha256 used for RSAES-OAEP-DECRYPT and, for example, sha1 used for MGF1. And if that's the case then what function is sha256 supposed to be used for? And what hash algorithm is supposed to be used for the other function?

And what does ECB mean in this context? ECB is a symmetric block cipher mode. Electronic Code Book. Maybe it's supposed to mean how Java deals with plaintext's that are larger than the modulo? Like maybe splits the plaintext into chunks that are as big as the modulo and then encrypts each one with RSA and concatenates them together? I'm just guessing..

Community
  • 1
  • 1
neubert
  • 15,947
  • 24
  • 120
  • 212

1 Answers1

55

OAEP uses a separate hash invocation to hash a (usually empty) label as well as a parameter to MGF1 (mask generation function), used for most of the OAEP padding.

The hash doesn't have that much impact on the security of OAEP, and for that reason it may be left to this default. Most libraries however use the same hash algorithm for MGF-1 and the hashing of the (always empty) label. Java however defaults to MGF1.

We can easily test this by comparing the standard Java Cipher instantiated using "OAEPWITHSHA-256ANDMGF1PADDING" against one instantiated using "OAEPPadding" and OAEPParameterSpec:

// --- we need a key pair to test encryption/decryption
KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
kpg.initialize(1024); // speedy generation, but not secure anymore
KeyPair kp = kpg.generateKeyPair();
RSAPublicKey pubkey = (RSAPublicKey) kp.getPublic();
RSAPrivateKey privkey = (RSAPrivateKey) kp.getPrivate();

// --- encrypt given algorithm string
Cipher oaepFromAlgo = Cipher.getInstance("RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING");
oaepFromAlgo.init(Cipher.ENCRYPT_MODE, pubkey);
byte[] ct = oaepFromAlgo.doFinal("owlstead".getBytes(StandardCharsets.UTF_8));
    
// --- decrypt given OAEPParameterSpec
Cipher oaepFromInit = Cipher.getInstance("RSA/ECB/OAEPPadding");
OAEPParameterSpec oaepParams = new OAEPParameterSpec("SHA-256", "MGF1", new MGF1ParameterSpec("SHA-1"), PSpecified.DEFAULT);
oaepFromInit.init(Cipher.DECRYPT_MODE, privkey, oaepParams);
byte[] pt = oaepFromInit.doFinal(ct);
System.out.println(new String(pt, StandardCharsets.UTF_8));

The code will fail with a padding related exception if you substitute "SHA-256" for the MGF1 as parameter, showing that SHA-1 is indeed the default.

The reason why the long algorithm string is needed is compatibility with other Cipher algorithms. Code written for, for instance "RSA/ECB/PKCS1Padding" doesn't use any parameters; without the longer string OAEP can therefore not function as drop in replacement.

The mode of operation "ECB" doesn't mean anything in this context, it should have been "None" or it should have been left out completely. You can only encrypt a single block using the RSA implementation of the SunRSA provider.

If you want to encrypt more data, create a random (AES) symmetric key and encrypt that using OAEP. Then use the AES key to encrypt your specific data. This is called a hybrid cryptosystem as it uses both asymmetric and symmetric primitives to encrypt data.

Note that OAEP is not supported in JDK 7 (1.7) or earlier. OAEP is included in the implementation requirements for Java runtimes since Java 8:

  • RSA/ECB/OAEPWithSHA-1AndMGF1Padding (1024, 2048)
  • RSA/ECB/OAEPWithSHA-256AndMGF1Padding (1024, 2048)

Some protocols may require you to use SHA-256 or SHA-512 within the padding, as SHA-1 is being deprecated for most use - even if it is not directly vulnerable for this kind of purpose. Neither SHA-224 or SHA-384 make any sense as those are versions of SHA-256 and SHA-512 with a reduced output size, and therefore require more calls to create the padding, without providing any security benefit (and yes, testing this does show a performance disadvantage).

If you have an invalid OAEP ciphertext you should first make sure that the right "default" is being used for the label and MGF1.

It is impossible to wrong any library implementation for choosing their own default; in the end it is up to the protocol to define the hashes used.

Unfortunately no mandatory default exists - which is especially a problem if protocol owners forget to fully specify a configuration for the algorithms.

Maarten Bodewes
  • 90,524
  • 13
  • 150
  • 263
  • 5
    I was playing around with the BouncyCastle crypto provider and BouncyCastle appears to operate differently with regard to `RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING`. ie. while Java's default crypto provider uses sha1 as the MGF hash bouncycastle appears to use sha256. This should probably be independently verified but that's the way it appears to me.. – neubert Sep 17 '16 at 05:04
  • 5
    I just verified this, sun.security.rsa.RSAPadding when used with OAEP SHA256, uses SHA1 for the MGF1 function, bouncy castle uses SHA256 for both, that's why they are not compatible. – peceps Nov 16 '16 at 21:36
  • @MaartenBodewes I also had a look at OpenSSL implementation, there SHA256 is not currently supported but it can be with a simple modification. There also the same hash function is used for both, so if the modification is made, it is compatible with bouncy castle. – peceps Nov 21 '16 at 15:27
  • 3
    WebCrypto uses it for both too. Thanks - this had me stumped. – rich Dec 07 '18 at 09:18
  • 1
    The code example above works only with bouncy castle provider. SunJCE does not support padding "OAEPPadding" (at least not JDK7). Maybe you add this as side-note. – Heri Apr 04 '19 at 09:44
  • @Heri Would something like "**don't use runtimes that are out of support for almost 4 years and counting, especially not for security related applications**" suite you? Seriously though, I'll make some kind of note about OAEP support. By the way: are you sure? [This document](https://docs.oracle.com/javase/7/docs/technotes/guides/security/StandardNames.html#Cipher) does list the algorithm. – Maarten Bodewes Apr 04 '19 at 11:19
  • 1
    @MaartenBodewes: Of course you are right. But you should not underestimate the laziness of the industry ("never change a running system"). I have no influence what java version our customers apply. And: yes I am sure: a ´´Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPPadding")´´ yields a NoSuchAlgorithmException both in Oracle 1.7.0_75 as in OpenJDK 1.7.0-u80. Only after adding BC-Provider the cipher gets initialized. – Heri Apr 09 '19 at 07:11
  • @MaartenBodewes: BTW: The section "Implementation Requirements" of your linked official document (link see above) does not list this transformation string, although the "Cipher Algorithm Padding" does. – Heri Apr 09 '19 at 07:31
  • The implementation requirements section is rather minimal, the Sun / Oracle providers provide way more algorithms than that. Sometimes however the spec. precedes the implementation (like it should). Even then, the algorithms often get back-ported. But, it seems, not in this case. Anyway, I guess with the comments it is clear that JDK 1.7 is not supported. Gotta run, will still adjust the answer text, thanks for the info. – Maarten Bodewes Apr 09 '19 at 08:32
  • I spent the whole day trying to understand why I couldn't get my payload to decrypt in WebCrypto. Using BouncyCastle works! Such an obscure and hard to catch error. – GuiSim May 16 '19 at 13:19
  • @MaartenBodewes Why do you say "Note that the hash chosen doesn't have that much impact on the security of OAEP"? – mad_fox Sep 20 '19 at 14:56
  • @mad_fox Basically you're not worried about collision resistance or length extension attacks within MGF1 (the function used for the padding internally), so you'd get the full 160 bits + of security even when using SHA-1. Of course, you'd want to use an unbroken hash function throughout your code as attacks get better, but security wise you should not be overly worried. If one of your runtimes only supports SHA-1 then it might be beneficial to stick to that. – Maarten Bodewes Sep 20 '19 at 15:06
  • @MaartenBodewes I took a look at this paper https://eprint.iacr.org/2006/223.pdf to better understand why I should not be worried about collision resistance. Based on that, my understanding is that the only requirement of the hashing algorithm used is that "the masking hash function is fully zero with only a negligible probability". To simplify, it means that if the chances of the hash being generated are almost zero, the hashing algorithm is allowed. So the odds of this are something like 1/2^160. Does that sound about right? – mad_fox Sep 20 '19 at 15:24
  • The paper states "Despite these pathological examples, we conjecture that the security properties necessary for the hash function are very minimal. Notably, our conjecture implies that collision resistance and preimage resistance are not actually necessary properties for the hash function in RSA-OAEP" which is great because SHA-1 is broken when it comes to collision resistance. – Maarten Bodewes Jan 09 '23 at 11:35
  • SHA-1 is default for the MGF1 for oracle, https://docs.oracle.com/javase/7/docs/api/index.html?javax/crypto/spec/OAEPParameterSpec.html What is the point? – Sam Ginrich Aug 11 '23 at 17:57