1

I have a function that parses a private RSA key using openssl's d2i_RSAPrivateKey function. I am finding that this succeeds with certain keys, but not others, even though all of these keys are PEM-encoded RSA keys. I generated my own signed public and private key, like this:

openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 36500 -nodes -outform PEM

I combined key.pem and cert.pem into a single file, combined.pem, and verified that I can parse the file with openssl's x509 and rsa commands:

openssl x509 -inform PEM -modulus -in combined.pem 
Modulus=B59A...

openssl rsa -inform PEM -modulus -in combined.pem 
Modulus=B59A...

But d2i_RSAPrivateKey silently fails to parse this file in code, returning an error code without printing anything. I know I'm using d2i_RSAPrivateKey correctly, because it works on a different public/private cert file. Can anyone tell why it can't parse the file I'm creating here on the command line?

jww
  • 97,681
  • 90
  • 411
  • 885
David Lobron
  • 1,039
  • 2
  • 12
  • 21
  • The intermittent failure is likely due to a ***`Subject Public Key Info`*** versus just a ***`Public Key`*** (the same apples to the private key). See, for example, [Convert PEM traditional private key to PKCS8 private key](http://stackoverflow.com/q/8290435). If it has an OID, then its a ***`Subject Public Key Info`*** (or private key equivalent). If it lacks an OID, then its just a ***`Public Key`*** (or private key equivalent). You can see the difference in the PEM. One will use `BEGIN PRIVATE KEY` (the one with OID), the other will use `BEGIN RSA PRIVATE KEY` (the one without an OID). – jww Oct 07 '15 at 22:43
  • Thanks! I checked and found that both files have "-----BEGIN PRIVATE KEY-----". I ran the command in the page you linked to (openssl pkcs8 -topk8 -inform PEM -outform PEM -in filename -out filename -nocrypt), changing DER to PEM for the outform, and the output was identical to the input. Is there anything else I could check here? – David Lobron Oct 08 '15 at 15:16

1 Answers1

2

After a thread with the openssl developers, it turned out that I was using the wrong function. I switched from d2i_RSAPrivateKey to d2i_PrivateKey (after extracting the EVP_PKEY with d2i_PrivateKey), and it works. I also switched from SSL_CTX_use_RSAPrivateKey_ASN1 to SSL_CTX_use_PrivateKey, again passing in the EVP_PEY, and it works.

One of the openssl developers suggested that my problem occurred because "the DER encoding of the RSAPrivateKey is what's contained in the long OCTET STRING element of the PKCS8_PRIV_KEY_INFO. You need to double-decode to get an actual RSAPrivateKey structure, whether by hand or via some other library function." I have not tested that yet, but I wanted to at least post this work-around solution.

David Lobron
  • 1,039
  • 2
  • 12
  • 21