2

I'm having trouble using RSACng (512) with OaepSHA256 (C#, .NET Framework 4.6.1) - getting System.Security.Cryptography.CryptographicException: 'The parameter is incorrect.' while encoding (any) byte array:

using System.Security.Cryptography;
...

using (RSACng rsaCng = new RSACng(512))
{
    var result = rsaCng.Encrypt(new byte[1], RSAEncryptionPadding.OaepSHA256); // throws ex
}

System.Security.Cryptography.CryptographicException: 'The parameter is incorrect.'
This exception was originally thrown at this call stack:
    System.Security.Cryptography.NCryptNative.EncryptData<T>(Microsoft.Win32.SafeHandles.SafeNCryptKeyHandle, byte[], ref T, System.Security.Cryptography.AsymmetricPaddingMode, System.Security.Cryptography.NCryptNative.NCryptEncryptor<T>)
    System.Security.Cryptography.NCryptNative.EncryptDataOaep(Microsoft.Win32.SafeHandles.SafeNCryptKeyHandle, byte[], string)
    System.Security.Cryptography.RSACng.Encrypt(byte[], System.Security.Cryptography.RSAEncryptionPadding)
    MiniJavaCertTest.Program.Test() in Program.cs
    MiniJavaCertTest.Program.Main() in Program.cs

If I use 1024 key size, it works:

using (RSACng rsaCng = new RSACng(1024))
{
    var result = rsaCng.Encrypt(new byte[1], RSAEncryptionPadding.OaepSHA256); // works ok
}

It also works if 512 key size is used, but with OaepSHA1 instead of OaepSHA256:

using (RSACng rsaCng = new RSACng(512))
{
    var result = rsaCng.Encrypt(new byte[1], RSAEncryptionPadding.OaepSHA1); // works ok
}

Any idea why doesn't this work and how to get it to work for 512 + OaepSHA256?

snejk85
  • 21
  • 2
  • I've been trying to solve this issue for a while. It appears there was a change in Net for the default encryption algorithm in Net 4.6 and Net 4.7. Many people have upgraded their working code to new Net and broke the encryption. Just found something interesting in c++ (https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/ns-wincrypt-crypt_algorithm_identifier) but not sure how to apply yet to c#. – jdweng Apr 29 '20 at 10:25
  • Tried it also in .NET Core (2.1 and 3.1) and the problem persists :( – snejk85 Apr 29 '20 at 12:14
  • What mode are you supposed to be using? Is it defined by a specification or do you have a certificate. Each encryption mode has valid and non valid options. You just can't pick any combination that you want. There are standards. See Wiki : https://en.wikipedia.org/wiki/Transport_Layer_Security – jdweng Apr 29 '20 at 13:41
  • For the moment I'm trying it only locally and this combination RSA 512 + OaepSHA256 is supposed to work and I got it to work in Java, but not in .NET. Gonna go with RSA 2048 (or 1024) in the end but would like to know is this a bug or not. – snejk85 Apr 29 '20 at 13:54
  • The padding mode is different in java from c#. What padding mode did you use? Padding adds random bits at end of key to hide true key size. Sp check number of bits in the key that works with java. If it is a standard size then it had padding. – jdweng Apr 29 '20 at 14:36

0 Answers0