-1

I have a file to be signed by following command:

$ openssl sha256 -sign privatekey.pem myfile.txt > right.result

Is there any equivalent command to get the same result? I have tried the following method (calculate the sha256 digest firstly, then sign the digest). But it can not work:

$ openssl sha256 -binary myfile.txt > my.digest
$ openssl rsautl -sign -inkey privatekey.pem -in my.digest > wrong.result

The content of "wrong.result" is always different from "right.result". Could someone tell me the reason? Thank you very much!

progquester
  • 1,228
  • 14
  • 23
  • 1
    Dupe https://stackoverflow.com/questions/38767660/multiple-openssl-rsa-signing-methods-produce-different-results/38768455 and others linked there – dave_thompson_085 Aug 29 '17 at 18:48

1 Answers1

-1

Using openssl sha256 -sign ... makes openssl compute the PKCS#1 "SHA256withRSA" algorithm, fully specified by RFC-8017.

This does not simply means doing a SHA-256 digest and sign it with RSA, as you did in the second part of your example.

Here is why:

According to this RFC, the signature must have the same length than the RSA modulus. Your RSA modulus is 512 bits long (default value using openssl genrsa) or more, so the computed signature MUST have a length of 512 bits (or more). Even if the hash function (HMAC-SHA256) returns values of 256 bits length.

So, to compute a signature with 512 bits, from a 256 bits only HMAC-SHA256 digest, RFC-8017 requires you to use the I2OSP Integer-to-Octet-String primitive, that takes 2 arguments: your 256 bits hash, and the integer 512 (the length of your RSA modulus). I2OSP will output a 512 bits length octet string. finally, this is this octets string that will be encrypted with your RSA private key.

But you have not computed this intermediate Integer-to-Octet-String value.This is why your signatures do not match.

Alexandre Fenyo
  • 4,526
  • 1
  • 17
  • 24