OpenSSL unable to load Public Key

Question

Trying to encrypt a text message via command line on OSX Yosomite 10.10.2

Created public .pem key like this:

ssh-keygen -f ~/.ssh/id_rsa.pub -e -t PKCS8 > id_rsa.pem

If I try to encrypt myMessage.txt

openssl rsautl -encrypt -inkey ~/.ssh/id_rsa.pem -pubin -in ~/Desktop/myMessage.txt -out ~/Desktop/encrypted.txt

I get unable to load Public key

If I then type:

openssl asn1parse -in id_rsa.pem

Returns: Error: offset too large

But I have no idea how to fix it. What should I change to make it work?

Answer 1

Still don't know what went wrong in my question but found a solution:

1. Generate RSA key:

   $ openssl genrsa -out key.pem 1024 
   $ openssl rsa -in key.pem -text -noout
   

2. Save public key in pub.pem file:

   $ openssl rsa -in key.pem -pubout -out pub.pem 
   $ openssl rsa -in pub.pem -pubin -text -noout 
   

3. Encrypt some data:

   $ echo test test test > file.txt 
   $ openssl rsautl -encrypt -inkey pub.pem -pubin -in file.txt -out file.bin 
   

4. Decrypt encrypted data:

   $ openssl rsautl -decrypt -inkey key.pem -in file.bin 
   

It works like a charm

Answer 2

I faced this problem also and think a good hint is here:

How can I transform between the two styles of public key format, one "BEGIN RSA PUBLIC KEY", the other is "BEGIN PUBLIC KEY"

It seems that the OpenSSL encryption command wants a SSL public key instead of a RSA public key.

We now know enough to tweak the example to make it work. A SSL public key can be generated from a RSA public key with

openssl rsa -in id_rsa.pem -RSAPublicKey_in -pubout > id_pub.pem

It is then possible to do the encryption step with

openssl rsautl -encrypt -inkey id_pub.pem -pubin -in ~/Desktop/myMessage.txt -out ~/Desktop/encrypted.txt

The default OpenSSL command in MacOSX Yosemite as of this writing appears to be 0.9.8zg. The rsa command in this version does not support the capability to run the first command above. I worked around this by installing OpenSSL 1.0.1p.

Answer 3

I had same problem when I was extracting public key from certificate.

openssl x509 -pubkey -noout -in cert.crt > pubKey.pem

Afterwards, I wanted to print information about key with command below.

openssl rsa -text -pubin -in pubKey.pem

And gets an error: unable to load Public Key

Solution

I opened pubKey.pem in notepad++ and in the Encoding menu was UCS-2 LE BOM selected. So I changed it to UTF-8 encoding. Size of pubKey.pem was half of the original one after changing encoding. Then it works like charm.

Tested in Windows and powershell

Answer 4

Your initial solution should work you just have a small typo: To specify key format (PKCS8), the "-m" option is used and not "-t" option (it stand for type of key: dsa, ecdsa, ed25519 or rsa). See ssh-keygen man page.

ssh-keygen -f ~/.ssh/id_rsa.pub -e -m PKCS8 > id_rsa.pem

Then, you could encrypt using this:

openssl rsautl -encrypt -inkey ~/.ssh/id_rsa.pem -pubin -in ~/Desktop/myMessage.txt -out ~/Desktop/encrypted.txt

And, you could decrypt using:

openssl rsautl -decrypt -inkey ~/.ssh/id_rsa -in ~/Desktop/encrypted.txt -out ~/Desktop/decrypted.txt

You could check diffrence between original and decrypted files using text editor or this diff command:

diff ~/Desktop/myMessage.txt ~/Desktop/decrypted.txt

In fact, openssl rsautl -encrypt command expect a public key with "PEM PKCS8 public key" encoding format but ssh-keygen generate a private key in this format and public key in other format adapted to authorized_keys file in ~/.ssh directory (you could open keys with text editor to see difference between formats).

Answer 5

I had to struggle with this subject on OSX (Ventura)

• permissions (bad permissions, Could not read... )
• wrong format (PEM/PKCS8 vs "SSH-type")

i came with this solution

# generate a PRIVATE KEY
openssl genpkey -algorithm RSA -aes-256-cbc -outform PEM -out private_key.pem -pkeyopt rsa_keygen_bits:2048
# create a passphrase made of 4 chars minimum

# SUDO ! change permissions to 600 at most
sudo chmod 600 private_key.pem
# but 400 should work and is event more secure
sudo chmod 400 private_key.pem

# SUDO again
# extract THE PUBLIC KEY in PKCS8 file format
sudo ssh-keygen -e -f private_key.pem -m PKCS8 > public_key.pem
# enter your passphrase

give the public key to someone who wants to protect the data.in.txt file. She or he will do the following to encrypt it:

# SUDO again
sudo openssl pkeyutl -encrypt -pubin -inkey public_key.pem -in data.in.txt -out data.enc.txt

You receive the encrypted file (data.enc.txt) to be decrypted:

# SUDO again
sudo openssl pkeyutl -decrypt -inkey private_key.pem -in data.enc.txt -out data.out.txt

# then you can read the confidential information
cat data.out.txt

you can do the same method so he/she extract his/her public key in order to encrypt your public key to protect it before sending it.