Check if SSH Private Key is Encrypted

Question

Key pairs generated with ssh-keygen on macOS can have different formats.

• The standard PEM ASN.1 object which is readable by macOS' SecKey APIs
• A PEM with textual headers
• OpenSSH Keys
• OpenSSH Encrypted Keys

OpenSSH/BSD uses this non-standardized format here.

Now I only need to be able to check if a private key has a passphrase set or not, so I can prompt the user to enter it, without having to deal with the complexities of different key formats.

Is there a quick way on macOS via Swift or C API, to check if a key has a passphrase set?

Answer 1

The difference between the unencrypted and encrypted private keys is the fact that the key blob is encrypted. You need to decrypt the private key blob data before you can use the private key blob. So once the encrypted private key data is decoded, you can treat it the same as the unencrypted private key data.

A unencrypted private key blob PEM file looks like this:

—–BEGIN PRIVATE KEY—–
{base64 private key blob)
—–END PRIVATE KEY—–

The encrypted RSA private key PEM file looks like this:

—–BEGIN RSA PRIVATE KEY—–
Proc-Type: 4,ENCRYPTED
DEK-Info: {encryption algorithm},{salt}
{base64 encrypted private key blob)
—–END RSA PRIVATE KEY—–

e.g.

—–BEGIN RSA PRIVATE KEY—–
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-256-CBC,AB8E2B5B2D989271273F6730B6F9C687
{base64 encrypted private key blob)
—–END RSA PRIVATE KEY—–

So to decode the private key data you need to:

1. Parse the DEK-Info encryption algorithm and the salt (good idea to confirm the first line is: "Proc-Type: 4,ENCRYPTED" as well).
2. Decode the base64 encrypted private key blob.
3. Generate the encryption algorithm "key" and "IV" based on the salt and the passphrase
4. Decode the encrypted private key blob.

Once you have done that the decrypted private key blob can be treated just like the unencoded private key blob.

The number of supported encryption algorithm's are rather large, so you may like to support a sub-set of algorithms. e.g. "DES-EDE3-CBC", "AES-xxx-CBC", etc

To generate the IV you need to convert salt string to binary. The salt string is a hex encoded string, so convert each two strings characters into a byte using a hex string character to byte converter.

For the generation of the encryption algorithm key you need the key size (e.g. DES-EDE3-CBC is 192bits, AES-256-CBC is 256bits). Build up the key "bits" with a loop appending MD5 hash results to the key until generate all the key bits required.

The MD5 HASH loop generation will consist of:

1. First MD5 Hash: MD5 hash of the first 8 bytes of the IV and the Passphrase
2. All other MD5 Hashes is the MD5 hash of the last MD5 hash result and the first 8 bytes of the IV and the Passphrase

See the openssl source for EVP_BytesToKey method for an example of the key bits generation.

The encrypted private key blob can now be decoded using the selected encryption algorithm using the IV and KEY build above.

Answer 2

There are two ways that I would suggest. Either reading the command line output using readLine() and checking if it asks for a password then do something accordingly.

import Foundation

func runCommand(cmd : String, args : String...) -> (output: [String], error: [String], exitCode: Int32) {

    var output : [String] = []
    var error : [String] = []

    let task = Process()
    task.launchPath = cmd
    task.arguments = args

    let outpipe = Pipe()
    task.standardOutput = outpipe
    let errpipe = Pipe()
    task.standardError = errpipe

    task.launch()

    let outdata = outpipe.fileHandleForReading.readDataToEndOfFile()
    if var string = String(data: outdata, encoding: .utf8) {
        string = string.trimmingCharacters(in: .newlines)
        output = string.components(separatedBy: "\n")
    }

    let errdata = errpipe.fileHandleForReading.readDataToEndOfFile()
    if var string = String(data: errdata, encoding: .utf8) {
        string = string.trimmingCharacters(in: .newlines)
        error = string.components(separatedBy: "\n")
    }

    task.waitUntilExit()
    let status = task.terminationStatus

    return (output, error, status)
}

//Sample usage

let (output, error, status) = runCommand(cmd: "/usr/local/bin/node", args: "--version")
print("program exited with status \(status)")
if output.count > 0 {
    print("program output:")
    print(output)
    //HERE YOU CAN CHECK IF PASSWORD REQUEST HAS BEEN MADE
}
if error.count > 0 {
    print("error output:")
    print(error)
}

The example code will return your node version installed if there is one, but you could use it to check if a password prompt has been made by the system for the RSA Key.

The other way could be perhaps using a third-party library like SwiftyRSA or BlueRSA which might help with validation.

Answer 3

I implemented my own OpenSSH check for the 2 most common formats

• For one I'm checking the PEM headers for DEK-Info for Linux-style SSH PEMs
• For OpenSSH style keys I manually parse the format using the class below

import Foundation

private let opensshMagic = "openssh-key-v1"

public class SSHPrivateKey {

    public struct OpenSSHKey {
        let cipherName: String
        let kdfName: String
        let kdfOptions: Data
        let numKeys: Int

        var isEncrypted: Bool {
            return cipherName != "none"
        }
    }

    public let data: Data

    public init(data: Data) {
        self.data = data
    }

    public func openSSHKey() -> OpenSSHKey? {
        // #define AUTH_MAGIC      "openssh-key-v1"
        //
        // byte[]  AUTH_MAGIC
        // string  ciphername
        // string  kdfname
        // string  kdfoptions
        // int     number of keys N
        // string  publickey1
        // string  publickey2
        // ...
        // string  publickeyN
        // string  encrypted, padded list of private keys

        guard let magic = opensshMagic.data(using: .utf8) else {
            return nil
        }

        if data.prefix(magic.count) != magic {
            return nil
        }

        var offset = magic.count + 1

        guard let cipherName = data.consumeString(offset: &offset),
            let kdfName = data.consumeString(offset: &offset) else {
                return nil
        }

        let kdfOptions = data.consumeBytes(offset: &offset)
        let numKeys = data.consumeUInt32(offset: &offset)

        return OpenSSHKey(cipherName: cipherName,
                          kdfName: kdfName,
                          kdfOptions: kdfOptions,
                          numKeys: Int(numKeys))
    }
}

private extension Data {

    func consumeBytes(offset: inout Int) -> Data {
        let n = Int(consumeUInt32(offset: &offset))
        let b = Data(self[offset..<offset + n])
        offset += n
        return b
    }

    func consumeString(offset: inout Int) -> String? {
        return consumeBytes(offset: &offset).utf8String
    }

    func consumeUInt8(offset: inout Int) -> UInt8 {
        let v = self[offset] & 0xFF
        offset += 1

        return v
    }

    func consumeUInt32(offset: inout Int) -> UInt32 {
        var i: UInt32 = 0

        i = (i << 8) | UInt32(consumeUInt8(offset: &offset))
        i = (i << 8) | UInt32(consumeUInt8(offset: &offset))
        i = (i << 8) | UInt32(consumeUInt8(offset: &offset))
        i = (i << 8) | UInt32(consumeUInt8(offset: &offset))

        return i
    }
}