31

I want to generate a RSA-SHA256 signature in Java, but I can't get it to produce the same signature as with OpenSSL on the console.

This is what I did with OpenSSL (following this tutorial):

Generate key pair:

openssl genrsa -out private.pem 1024

Extract public key:

openssl rsa -in private.pem -out public.pem -outform PEM -pubout

Create hash of data:

echo 'data to sign' > data.txt
openssl dgst -sha256 < data.txt > hash

The generated hash file starts with (stdin)= what I removed by hand (first forgot to mention it, thanks mata).

Sign hash:

openssl rsautl -sign -inkey private.pem -keyform PEM -in hash  > signature

To reproduce the results in Java I first converted the private key from PEM to DER:

openssl pkcs8 -topk8 -inform PEM -outform DER -in private.pem -nocrypt > private.der

Now I wrote this Java class to generate the same signature:

public class RSATest {

    public static void main(String[] args) throws IOException,
            NoSuchAlgorithmException, InvalidKeySpecException,
            InvalidKeyException, SignatureException {

        byte[] encodedPrivateKey = readFile("private.der");
        byte[] content = readFile("data.txt");

        KeyFactory keyFactory = KeyFactory.getInstance("RSA");
        PKCS8EncodedKeySpec keySpec = new PKCS8EncodedKeySpec(encodedPrivateKey);
        RSAPrivateKey privateKey = (RSAPrivateKey) keyFactory
                .generatePrivate(keySpec);

        Signature signature = Signature.getInstance("SHA256withRSA");
        signature.initSign(privateKey);
        signature.update(content);
        byte[] signatureBytes = signature.sign();

        FileOutputStream fos = new FileOutputStream("signature-java");
        fos.write(signatureBytes);
        fos.close();
    }

    private static byte[] readFile(String filename) throws IOException {
        File file = new File(filename);
        BufferedInputStream bis = new BufferedInputStream(new FileInputStream(
                file));
        byte[] bytes = new byte[(int) file.length()];
        bis.read(bytes);
        bis.close();
        return bytes;
    }

}

Unfortunately the results are not the same, so I think I must have done something wrong, but I can't figure out what. Can someone of you help me to find the bug?

Arthur Dent
  • 1,248
  • 2
  • 12
  • 23
  • Random padding and message encoding will ensure the signatures don't match. Maybe you should use a deterministic signature scheme? – jww Dec 04 '16 at 07:33

1 Answers1

48
openssl dgst -sha256 < data.txt

produces something like:

(stdin)= b39eaeb437e33087132f01c2abc60c6a16904ee3771cd7b0d622d01061b40729

notice the (stdin)='? you don't want that to be part of your hash, if you need to create a digest, use the -binary option.

try using this to sign your data:

openssl sha -sha256 -sign private.pem < data.txt

This does everything you need.


edit - a little more explanations:

let's create a digest and show it

$ openssl dgst -sha256 -binary < data.txt > digest
$ hd digest
00000000  26 3b 0a a1 2e b9 32 db  b8 dc d3 6f 37 94 0b 05  |&;....2....o7...|
00000010  71 9c ba 79 46 34 28 9f  5c 5b 98 9a 64 61 c9 ec  |q..yF4(.\[..da..|

now we take this digest and sign int using rsautl:

$ openssl rsautl -sign -inkey private.pem < digest > sign1
$ hd sign1
00000000  1b 7a cf a4 8d 41 8e 04  ed 3a bb ba 86 f1 f8 e0  |.z...A...:......|
00000010  df f7 47 3e d7 a7 f4 90  7a 05 f8 7f 45 e5 29 e7  |..G>....z...E.).|
00000020  9f f4 2c 91 97 2f e7 26  69 9f 6a 07 a3 48 1b 85  |..,../.&i.j..H..|
00000030  2e f8 ee 44 4d 25 9f ae  05 95 81 c9 e3 07 68 ad  |...DM%........h.|

now let's sign the same file using dgst directly:

$ openssl dgst -sha256 -sign private.pem < data.txt > sign2
$ hd sign2
00000000  15 c2 94 87 eb e6 cb 45  c8 63 0c 97 60 d3 07 f3  |.......E.c..`...|
00000010  dc 65 32 ad 44 1c c2 2a  7f a3 e1 fc dd 84 27 8c  |.e2.D..*......'.|
00000020  77 a6 97 2b 33 6b c6 d7  7d e1 1d 39 5c 48 b6 48  |w..+3k..}..9\H.H|
00000030  cb 18 be bf 6a 66 90 d3  88 89 52 6c dd d1 b9 99  |....jf....Rl....|

So what's different here? To see that, we can verify the signature and show the raw output. Both files do contain the digest, but the metadata and padding is different:

$ openssl rsautl -raw -verify -inkey private.pem < sign1 | hd
00000000  00 01 ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
00000010  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff 00  |................|
00000020  26 3b 0a a1 2e b9 32 db  b8 dc d3 6f 37 94 0b 05  |&;....2....o7...|
00000030  71 9c ba 79 46 34 28 9f  5c 5b 98 9a 64 61 c9 ec  |q..yF4(.\[..da..|

$ openssl rsautl -raw -verify -inkey private.pem < sign2 | hd
00000000  00 01 ff ff ff ff ff ff  ff ff ff ff 00 30 31 30  |.............010|
00000010  0d 06 09 60 86 48 01 65  03 04 02 01 05 00 04 20  |...`.H.e....... |
00000020  26 3b 0a a1 2e b9 32 db  b8 dc d3 6f 37 94 0b 05  |&;....2....o7...|
00000030  71 9c ba 79 46 34 28 9f  5c 5b 98 9a 64 61 c9 ec  |q..yF4(.\[..da..|

To see this more clearly, we can try to use the -asn1parse flag, which won't work for the first signature, but for the second it shows the correct structure of the signature:

$ openssl rsautl -verify -inkey private.pem -asn1parse < sign1
Error in encoding
139931349546656:error:0D07209B:asn1 encoding routines:ASN1_get_object:too long:asn1_lib.c:142:

$ openssl rsautl -verify -inkey private.pem -asn1parse < sign2
    0:d=0  hl=2 l=  49 cons: SEQUENCE          
    2:d=1  hl=2 l=  13 cons:  SEQUENCE          
    4:d=2  hl=2 l=   9 prim:   OBJECT            :sha256
   15:d=2  hl=2 l=   0 prim:   NULL              
   17:d=1  hl=2 l=  32 prim:  OCTET STRING      
      0000 - 26 3b 0a a1 2e b9 32 db-b8 dc d3 6f 37 94 0b 05   &;....2....o7...
      0010 - 71 9c ba 79 46 34 28 9f-5c 5b 98 9a 64 61 c9 ec   q..yF4(.\[..da..
mata
  • 67,110
  • 10
  • 163
  • 162
  • You're right, I already noticed it and removed '(stdin)= ' from the hash file by hand, but forgot to mention it in my question. I will correct my question. Thanks. – Arthur Dent Nov 16 '12 at 15:50
  • 1
    nevertheless, you should create the signature using the second command I posted, the way you're doing it you sign the string representation of the hash instead of the binary, and this doesn't really create a valid signature. see my answer from [here](http://stackoverflow.com/questions/11221898/m2crypto-rsa-sign-vs-openssl-rsautl-sign/11227055#11227055) – mata Nov 16 '12 at 16:05
  • You're right, again. Using your command the signature is the same as the one generated by Java. But another problem appears for me. My old (incorrect) signature was successfully verified with the command `openssl rsautl -verify -inkey public.pem -keyform PEM -pubin -in signature > verified`, what does not work anymore. Can you tell me what the correct command for verification is? – Arthur Dent Nov 16 '12 at 16:24
  • 1
    I've tried to explain it by showing some examples. I hope it makes it clearer. – mata Nov 16 '12 at 16:51
  • Big +1 from me for the padding point out. Can you explain, why there is the difference betweeen using the two ways of generating signature? Is the `openssl rsautl -sign` generating invalid output? – kamituel Nov 28 '14 at 08:23
  • 2
    In the first case you take an arbitrary digest and sign it, the information that the digest was created using sha256 is lost in this process, so the resulting signature won't/can't contain that information, whereas the ASN1 data from the second signature shows that it does. So the first signature could only be verified if the verifier knows exactly how the digest was procduced, so its not really invalid but incomplete (lower level) output. – mata Nov 28 '14 at 10:51
  • Great answer, thanks @mata! I refered to your answer [in the blog post I wrote](http://kamituel.tumblr.com/post/104753711213/using-android-keystore-in-the-javascript-app-with), I hope it'll give you many upvotes! – kamituel Dec 09 '14 at 13:18
  • Isn't there a typo in the second command, `openssl sha ...` - shouldn't it be `openssl dgst ...` ? – Juraj Martinka Jan 05 '23 at 20:55
  • @JurajMartinka On older openssl versions there was a `sha` subcommand - see: https://www.openssl.org/docs/man1.0.2/man1/openssl.html The functionality was merged into the dgst subcommand at some point... – mata Jan 22 '23 at 18:14