Trying to wrap my head around signing and use/test various options.
I can sign using this command:
openssl dgst -sha256 -sign private_key.pem -binary -out sig_file data_file
But the documentation seems to say that I can also use this method
openssl dgst -sha256 -binary data_file > hash_file
openssl rsautl -sign -inkey private_key.pem -keyform PEM -in hash_file > sig_file2
But the signatures are different when I'd expect them to be identical. Either I missed something in the options or something else is wrong in my assumptions.
The real question from this issue: Is there a way to sign using command line options given that I already have the hash value and produce a signature that is identical to the first command above.
To add more to this, I can reproduce the first command easily in code and it matches the first command above meaning that I can sign with the hash value calculated first.
mdctx = EVP_MD_CTX_create();
EVP_DigestInit_ex(mdctx, EVP_sha256(), NULL);
EVP_DigestUpdate(mdctx, data, len);
EVP_DigestFinal_ex(mdctx, hash, &s);
EVP_MD_CTX_destroy(mdctx);
kfile = fopen64(key_file, "r");
key = PEM_read_RSAPrivateKey(kfile, NULL, NULL, NULL);
fclose(kfile);
*sig = malloc(RSA_size(key));
RSA_sign(NID_sha256, hash, hlen, *sig, siglen, key);