Decrypt with private key file: invalid stream header

Question

In command line I can encrypt and decrypt my file with openssl as following:

Create Keys:

openssl req -x509 -newkey rsa:2048 -keyout myKey.key -out myKey.crt -pubkey

Encrypt:

cat myText.txt | openssl rsautl -encrypt -inkey myKey.crt -pubin >> encryptedtxt.enc

Decrypt:

openssl rsautl -decrypt -inkey myKey.key -in encryptedtxt.enc > decryptedtxt.txt

I followed the this tutorial

public static void main(String[] args) {

        String encryptedData = "..\\encryptedtxt.enc";
        File encryptedFIle = new File(encryptedData);

        try {
            byte[] data = Files.readAllBytes(encryptedFIle.toPath());
            PrivateKey privateKey = getPrivateKey();
            Cipher cipher = Cipher.getInstance("RSA");
            cipher.init(Cipher.DECRYPT_MODE, privateKey);
            byte[] decryptedData = cipher.doFinal(data);
            System.out.println(">" + new String(decryptedData));
        } catch (Exception e) {
            e.printStackTrace();
        }
    }


    private static PrivateKey getPrivateKey() {
        String privateKeyFilename = "\\myKey.key";
        FileInputStream fis = null;
        ObjectInputStream ois = null;
        File privateKeyFile = new File(privateKeyFilename);
        try {
            fis = new FileInputStream(privateKeyFile);
            ois = new ObjectInputStream(fis);
            BigInteger modulus = (BigInteger) ois.readObject();
            BigInteger exponent = (BigInteger) ois.readObject();
            RSAPrivateKeySpec rsaPrivateKeySpec = new RSAPrivateKeySpec(modulus, exponent);
            KeyFactory fact = KeyFactory.getInstance("RSA");
            PrivateKey privateKey = fact.generatePrivate(rsaPrivateKeySpec);
            return privateKey;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

When this is run, following issue occurs:

java.io.StreamCorruptedException: invalid stream header: 2D2D2D2D
    at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:857)
    at java.io.ObjectInputStream.<init>(ObjectInputStream.java:349)
    at Decryptor.getPrivateKey(TestClass.java:38)
    at Decryptor.main(TestClass.java:20)
java.security.InvalidKeyException: No installed provider supports this key: (null)
    at javax.crypto.Cipher.chooseProvider(Cipher.java:893)
    at javax.crypto.Cipher.init(Cipher.java:1249)
    at javax.crypto.Cipher.init(Cipher.java:1186)
    at Decryptor.main(TestClass.java:22)

Any suggestion please how I can I solve this?

UPDATE:

I modified my getting private key method as following:

private static PrivateKey getPrivateKey() {
        String privateKeyFilename = "myKey.key";
        FileInputStream fis = null;
        ObjectInputStream ois = null;
        File privateKeyFile = new File(privateKeyFilename);
        try {
            String key = readFileAsString(privateKeyFilename);
            BASE64Decoder b64 = new BASE64Decoder();
            byte[] pkcs8EncodedBytes = b64.decodeBuffer(key);

            PKCS8EncodedKeySpec keySpec = new PKCS8EncodedKeySpec(pkcs8EncodedBytes);
            KeyFactory kf = KeyFactory.getInstance("RSA");
            PrivateKey privKey = kf.generatePrivate(keySpec);
            System.out.println(privKey);
            return privKey;

        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

But then the following error is thrown:

java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: IOException : Short read of DER length
    at sun.security.rsa.RSAKeyFactory.engineGeneratePrivate(RSAKeyFactory.java:217)
    at java.security.KeyFactory.generatePrivate(KeyFactory.java:372)
    at Decryptor.getPrivateKey(TestClass.java:61)
    at Decryptor.main(TestClass.java:19)
Caused by: java.security.InvalidKeyException: IOException : Short read of DER length
    at sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:351)
    at sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:356)
    at sun.security.rsa.RSAPrivateCrtKeyImpl.<init>(RSAPrivateCrtKeyImpl.java:91)
    at sun.security.rsa.RSAPrivateCrtKeyImpl.newKey(RSAPrivateCrtKeyImpl.java:75)
    at sun.security.rsa.RSAKeyFactory.generatePrivate(RSAKeyFactory.java:316)
    at sun.security.rsa.RSAKeyFactory.engineGeneratePrivate(RSAKeyFactory.java:213)
    ... 3 more
java.security.InvalidKeyException: No installed provider supports this key: (null)
    at javax.crypto.Cipher.chooseProvider(Cipher.java:893)
    at javax.crypto.Cipher.init(Cipher.java:1249)
    at javax.crypto.Cipher.init(Cipher.java:1186)
    at Decryptor.main(TestClass.java:21)

If you dont use BouncyCastle, standart jre requires a patch (jce) to handle key size larger than 128 bits. It usually throws different exception(indicating jce is not installed). But in order to do it with plain jre you should install jce if you haven't. Maybe it will fix this. — miskender, Feb 23 '18 at 18:02

score 3 · Accepted Answer · answered Feb 23 '18 at 11:13

You are trying to read the key using an ObjectInputStream. This class is not meant for general purpose decoding; it only decodes a Java-specific serialization format. The error you are seeing is the ObjectInputStream informing you that the data that you are reading is not a serialized Java object.

The key file generated by OpenSSL is not a Java-serialized object. Instead, it uses PEM encoding. For more information on reading keys from a PEM file, have a look at Decrypting an OpenSSL PEM Encoded RSA private key with Java?

I modified the getting private key method with base64: but still it fails. I updated my question too. — Débora, Feb 23 '18 at 12:44

Decrypt with private key file: invalid stream header

1 Answers1