why nodejs crypto sign function only accept privateKey pem format?

Question

at first,we use elliptic crypto package . the sign function looks like this:

signByPriv = function (privKeyData, text) {
    let msgHash = getmsgHash(text, "SHA-384");
    let key = ec.keyFromPrivate(Buffer.from(privKeyData,'base64').toString('hex'), 'hex')
    let signature = key.sign(msgHash);
    return signature
}

then we wanna change it to nodejs version,cause nodejs use openssl under the hood。so it will be faster

at first my sign function as below:

signByPriv = function (privKeyData, text) {
    const sign1 = crypto.createSign('SHA384');  //hash do inside
    sign1.write(text);
    sign1.end();
    const signature = sign1.sign(privKeyData, 'hex');
    return signature;
}

it will complain about the error:

internal/crypto/sig.js:86 const ret = this[kHandle].sign(data, format, type, passphrase, rsaPadding,

Error: error:0909006C:PEM routines:get_name:no start line

so I checked the nodejs docs,and found it need to pass the privKey with pem format。

signByPriv = function (privKeyData, text) {
    let key = turnBase64PrivToPemKey(privKeyData) //base64 => pem
    const sign1 = crypto.createSign('SHA384');  //hash do inside
    sign1.write(text);
    sign1.end();
    const signature = sign1.sign(privKeyData, 'hex');
    return signature;
}
turnBase64PrivToPemKey = function (base64Priv) {
    var key_hex = Buffer.from(base64Priv, 'base64').toString('hex');
    ecdh.setPrivateKey(key_hex, 'hex')
    var pubKey_hex = ecdh.getPublicKey().toString('hex');
    //pem格式私钥文件是由固定字符加上私钥和公钥拼接而成==同一条曲线，固定字符相同
    var mykey = '308187020100301306072a8648ce3d020106082a8648ce3d030107046d306b0201010420' + key_hex + 'a144034200' + pubKey_hex;
    privKey = '-----BEGIN PRIVATE KEY-----\n' + Buffer.from(mykey, 'hex').toString('base64') + '\n-----END PRIVATE KEY-----';
    pubKey = crypto.createPublicKey(privKey); //也可恢复出公钥
    let Key = {
        privKey,
        pubKey
    }
    return Key;
}

and great,the sign and verify functions all works perfect。

but the backend may do the same stupid thing...

the curve we chose is prime256v1

const ecdh = crypto.createECDH('prime256v1')

so,I wonder why nodejs sign func can't accept only a base64 priv?

cause the pem format is only composed by private key,public key and other fixed string.

score 10 · Accepted Answer · edited Oct 07 '21 at 08:14

Sign and Verify don't only support the PEM-format, but also the DER-format (both described in the answer of @Maarten Bodewes). Furthermore, both Pkcs8- (RFC 5208 and here) and Sec1-EC-keys (SECG SEC1, section C.4 and here) can be used. However, raw EC-keys are not directly supported. Therefore, if the keys are only available as raw keys, a conversion is always necessary. But this conversion is much easier to implement than in the posted code, so that (in my opinion) there is no significant additional effort, e.g. for signing:

var buf1 = Buffer.from('308141020100301306072a8648ce3d020106082a8648ce3d030107042730250201010420', 'hex'); // specific byte-sequence for curve prime256v1
var buf2 = Buffer.from('<Raw private key as hex string>', 'hex'); // raw private key (32 bytes)
var privateKeyPkcs8Der = Buffer.concat([buf1, buf2], buf1.length + buf2.length);
var sign = crypto.createSign('SHA384');
sign.write(<data to sign>);
sign.end();
var signature = sign.sign({ key: privateKeyPkcs8Der, format: 'der', type: 'pkcs8' }); // specify format and type

The key used for signing, privateKeyPkcs8Der, is a Pkcs8-key in DER-format (without raw public key).

In contrast to the posted code:

the DER-format is used instead of the PEM-format.
only the raw private key is embedded in the Pkcs8-container and not the raw public key.
buf1 contains the byte-sequence that belongs to prime256v1 (secp256r1) and a Pkcs8-container without raw public key. Note: The byte-sequence differs slightly from the byte-sequence used in the posted code. This is because the byte-sequence also contains length information and this is different depending on whether the raw public key is embedded or not.

The same applies to verification:

var buf1 = Buffer.from('3059301306072a8648ce3d020106082a8648ce3d030107034200', 'hex'); // specific byte-sequence for curve prime256v1
var buf2 = Buffer.from('<Raw public key as hex string>', 'hex'); // raw public key (uncompressed, 65 bytes, startting with 04)
var publicKeyX509Der = Buffer.concat([buf1, buf2], buf1.length + buf2.length);
var verify = crypto.createVerify('SHA384');    
verify.write(<data to sign>);
verify.end();
var verified = verify.verify({ key: publicKeyX509Der, format: 'der', type: 'spki' }, signature); // specify format and type

The key used for verification, publicKeyX509Der, is an X.509-SubjectPublicKeyInfo-key (SECG SEC1, section C.3) in DER-format.

As with signing:

the DER-format is used instead of the PEM-format.
buf1 contains the byte-sequence that belongs to prime256v1.

In the posted code, the methods of the ECDH-class are used to derive the raw public key from the raw private key. Instead, the createPublicKey- and export-method can be used to derive the X.509-SubjectPublicKeyInfo-key from the Pkcs8-key:

var publicKey = crypto.createPublicKey({ key: privKeyPkcs8DER, type: 'pkcs8', format: 'der' });
var publicKeyX509Der = publicKey.export({type: 'spki', format: 'der'})

Here privateKeyPkcs8Der is a Pkcs8-key (with or without raw public key) and publicKeyX509Der is an X.509-SubjectPublicKeyInfo-key, both in DER-format.

Note: A Sec1-container could also be used instead of the Pkcs8-container. However, then the structure of the key and the byte-sequences have to be adapted accordingly. The use of a Sec1-container is described here, but for a different curve (secp256k1), so that the byte-sequences can' t simply be copied.

yeah,I've tried your code and it all works。so the sign/verify function can't accept a raw priv key ... any sources to help me be more familar with these standards,thanks — John, Oct 14 '19 at 01:59
WOW this is one of the most valuable pieces of information on the web! It's a grain of gold. Thanks man. This simplifies the web-push signing process drastically since the asn1 dependency required to generate PAM can be removed. — zavr, Apr 17 '20 at 17:57

score 3 · Answer 2 · answered Oct 12 '19 at 10:46

PEM is a standard that is handled by OpenSSL. It defines header lines, a footer line and a specific type of base 64 and ways of handling data. The header line and footer line also clearly identifies the type of data encoded in base 64. It was initially mainly used for mail; PEM means privacy enhanced mail.

So these are advantages over base 64. You can clearly find the start and the end of the PEM, the type of the object contained in the PEM, etc. Note as well that PEM predates any official standard for base 64 to my knowledge. It of course existed before that, but each standard that required encoding / decoding simply re-defined it internally.

So yeah, that's how OpenSSL started to use PEM, and NodeJS as you mentioned uses that.

Note that many libraries also contain a base 64 decoder. Within the base 64 there is usually a binary format called DER containing data described by the ASN.1 data description language. If your library can handle DER then you can simply decode the base 64 yourself and use the ASN.1 functionality within your cryptography library. The OpenSSL command line can do this by the base64 command and the -inform DER switch.

why nodejs crypto sign function only accept privateKey pem format?

2 Answers2