Decrypt aes encrypted file in java sha1 openssl

Question

Im trying to implement file decryption by referring this code:

Encryption part i have done this way: https//stackoverflow.com/questions/64423926/encrypt-file-in-java-and-decrypt-in-openssl-with-key-aes

And the encrypted file im able to decrypt with openssl.

But the decrypt to file in java is causing error:

java.lang.IllegalArgumentException: Illegal base64 character 5f
    at java.util.Base64$Decoder.decode0(Unknown Source)
    at java.util.Base64$Decoder.decode(Unknown Source)
    at java.util.Base64$Decoder.decode(Unknown Source)
    at aes.AesEncryptTask.decryptNew(AesEncryptTask.java:107)
    at aes.AesEncryptTask.main(AesEncryptTask.java:58)

Content in my encrypted file is:

Salted__¨‹–1ž#¡ð=—ÖÏùá•NEÄ

Note: Starting salted part is not base64encoded. following data is encoded.

Please suggest on correct implementation of file decryption.

static void decryptNew(String path,String password, String outPath) {
        try{
        FileInputStream fis = new FileInputStream(path);
        FileOutputStream fos = new FileOutputStream(outPath);
        final byte[] pass = password.getBytes(StandardCharsets.US_ASCII);
        //final byte[] inBytes = Base64.getDecoder().decode(source);
        String source = getFileContent(fis);
        final Decoder decoder = Base64.getDecoder();
        final byte[] inBytes = decoder.decode(source);
        //final byte[] inBytes =source.getBytes();//DatatypeConverter.parseBase64Binary(source);
        final byte[] shouldBeMagic = Arrays.copyOfRange(inBytes, 0, SALTED_MAGIC.length);
        if (!Arrays.equals(shouldBeMagic, SALTED_MAGIC)) {
            throw new IllegalArgumentException("Initial bytes from input do not match OpenSSL SALTED_MAGIC salt value.");
        }
        final byte[] salt = Arrays.copyOfRange(inBytes, SALTED_MAGIC.length, SALTED_MAGIC.length + 8);
        final byte[] passAndSalt = array_concat(pass, salt);
        byte[] hash = new byte[0];
        byte[] keyAndIv = new byte[0];
        for (int i = 0; i < 3 && keyAndIv.length < 48; i++) {
            final byte[] hashData = array_concat(hash, passAndSalt);
            MessageDigest md = null;
            md = MessageDigest.getInstance("SHA-1");
            hash = md.digest(hashData);
            keyAndIv = array_concat(keyAndIv, hash);
        }

        final byte[] keyValue = Arrays.copyOfRange(keyAndIv, 0, 32);
        final SecretKeySpec key = new SecretKeySpec(keyValue, "AES");

        final byte[] iv = Arrays.copyOfRange(keyAndIv, 32, 48);

        final Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
        cipher.init(Cipher.DECRYPT_MODE, key, new IvParameterSpec(iv));
        final byte[] clear = cipher.doFinal(inBytes, 16, inBytes.length - 16);
        String contentDecoded = new String(clear, StandardCharsets.UTF_8);
        
        fos.write(contentDecoded.getBytes());    
        fos.close();
        
        //cipher.init(Cipher.DECRYPT_MODE, sks);
        /*CipherInputStream cis = new CipherInputStream(fis, cipher);
        int b;
        byte[] d = new byte[8];
        while((b = cis.read(d)) != -1) {
            fos.write(d, 0, b);
        }
        fos.flush();
        fos.close();
        cis.close();*/
        
        System.out.println("Decrypt is completed");
        }catch(Exception e){
            e.printStackTrace();
            
        }
    }

`Illegal base64 character 5f`, ie underscore is indeed not valid in Base64. Are you sure your input file is in fact Base64-encoded? — Gereon, Oct 20 '20 at 10:40
@Gereon, salt value in begining of my file is not encoded. decryption(openssl) is not happening if i encode salt value also. Currently my file content is like this: Salted__¨‹–1ž#¡ð=—ÖÏùá•NEÄ — Anees U, Oct 20 '20 at 11:34
@michalk, that also causing error java.lang.IllegalArgumentException: Illegal base64 character 18 — Anees U, Oct 20 '20 at 11:35
Your content is clearly not base64 encoded. You need to know exactly what you have and how it is encoded, it's not really a feasible plan to just guess random stuff until it appears to work. — rzwitserloot, Oct 20 '20 at 11:51
The link in your line "Encryption part i have done this way" isn't working so we cannot see how your encryption takes place. Kindly post a sample of simple plaintextfile and encrypted-file contents (best in hex code) - as @rzwitserloot commented it is definitely **not base64 encoded**. — Michael Fehr, Oct 20 '20 at 12:02
As mentioned before, the second link is currently not working, but based on the history you mean [this code](https://stackoverflow.com/q/64423926/9014097). There, the data is not Base64 encoded and as digest MD5 is applied. Also note concerning the code posted here, that because of the UTF8 decoding at the end only UTF8 encoded plaintexts can be decrypted (other data are corrupted). A big disadvantage of the design is that the data is read as an entire block (`getFileContent()`). It would be better (after reading the first block) to process the data in chunks (e.g. with `CipherInputStream`). — Topaco, Oct 20 '20 at 12:17
You referred to your yesterday's question that was successfully answered by @Topaco and the ciphertext is **NOT base64-encoded** so why do you try to decode the complete file? Leave out the base64-decoding, read in the salt (throwing away the magic "Salted__" string), generate the key and feed the rest of the ciphertext to your decryption function. As Topaco noted you should do this in chunks because when the file size rises you may run in "out of memory" error. — Michael Fehr, Oct 20 '20 at 12:21
@MichaelFehr, >>"read in the salt (throwing away the magic "Salted__" string), generate the key and feed the rest of the ciphertext to your decryption function." Sorry im confused about these. Kindly show a sample code. — Anees U, Oct 20 '20 at 15:23

score 0 · Accepted Answer · answered Oct 20 '20 at 17:24

The below code is doing a complete file encryption and decryption and is compatible to the OpenSSL commands

encrypt: openssl enc -aes-256-cbc -pass pass:testpass -d -p -in plaintext.txt -out plaintext.txt.crypt -md md5
decrypt: openssl aes-256-cbc -d -in plaintext.txt.crypt -out plaintext1.txt -k testpass -md md5

I left out some variables as they are not used in decryption method and on the other hand substituded the byte[] concatination with a pure Java version.

The simple output is:

encrypt done plaintext.txt
ciphertext: plaintext.txt
Decrypt is completed

Security warning: the key derivation is DEPRECATED and should not used any longer. The code does not have any exception handling and is for educational purpose only.

code:

import javax.crypto.Cipher;
import javax.crypto.CipherOutputStream;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.*;
import java.util.Arrays;

public class MainSo {
    public static void main(String[] args) {
        System.out.println("https://stackoverflow.com/questions/64443373/decrypt-aes-encrypted-file-in-java-sha1-openssl?noredirect=1#comment113960588_64443373");
        String plaintextFilename = "plaintext.txt";
        String ciphertextFilename = "plaintext.txt.crypt";
        String decryptedtextFilename = "plaintextDecrypted.txt";
        String password = "testpass";

        // openssl equivalent:
        // decrypt: openssl aes-256-cbc -d -in plaintext.txt.crypt -out plaintext1.txt -k testpass -md md5
        // encrypt: openssl enc -aes-256-cbc -pass pass:testpass -d -p -in sample.crypt -out sample.txt -md md5

        String ciphertext = encryptfile(plaintextFilename, password);
        System.out.println("ciphertext: " + ciphertext);
        decryptNew(ciphertextFilename, password, decryptedtextFilename);
    }

    public static String encryptfile(String path, String password) {
        try {
            FileInputStream fis = new FileInputStream(path);
            FileOutputStream fos = new FileOutputStream(path.concat(".crypt"));
            final byte[] pass = password.getBytes(StandardCharsets.UTF_8);
            final byte[] salt = (new SecureRandom()).generateSeed(8);
            fos.write("Salted__".getBytes(StandardCharsets.UTF_8));
            fos.write(salt);
            final byte[] passAndSalt = concatenateByteArrays(pass, salt);
            byte[] hash = new byte[0];
            byte[] keyAndIv = new byte[0];
            for (int i = 0; i < 3 && keyAndIv.length < 48; i++) {
                final byte[] hashData = concatenateByteArrays(hash, passAndSalt);
                final MessageDigest md = MessageDigest.getInstance("MD5");
                hash = md.digest(hashData);
                keyAndIv = concatenateByteArrays(keyAndIv, hash);
            }
            final byte[] keyValue = Arrays.copyOfRange(keyAndIv, 0, 32);
            final byte[] iv = Arrays.copyOfRange(keyAndIv, 32, 48);
            final SecretKeySpec key = new SecretKeySpec(keyValue, "AES");
            final Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
            cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
            CipherOutputStream cos = new CipherOutputStream(fos, cipher);
            int b;
            byte[] d = new byte[8];
            while ((b = fis.read(d)) != -1) {
                cos.write(d, 0, b);
            }
            cos.flush();
            cos.close();
            fis.close();
            System.out.println("encrypt done " + path);
        } catch (IOException | NoSuchAlgorithmException | NoSuchPaddingException | InvalidKeyException | InvalidAlgorithmParameterException e) {
            e.printStackTrace();
        }
        return path;
    }

    static void decryptNew(String path,String password, String outPath) {
        byte[] SALTED_MAGIC = "Salted__".getBytes(StandardCharsets.UTF_8);
        try{
            FileInputStream fis = new FileInputStream(path);
            FileOutputStream fos = new FileOutputStream(outPath);
            final byte[] pass = password.getBytes(StandardCharsets.US_ASCII);
            final byte[] inBytes = Files.readAllBytes(Paths.get(path));
            final byte[] shouldBeMagic = Arrays.copyOfRange(inBytes, 0, SALTED_MAGIC.length);
            if (!Arrays.equals(shouldBeMagic, SALTED_MAGIC)) {
                throw new IllegalArgumentException("Initial bytes from input do not match OpenSSL SALTED_MAGIC salt value.");
            }
            final byte[] salt = Arrays.copyOfRange(inBytes, SALTED_MAGIC.length, SALTED_MAGIC.length + 8);
            final byte[] passAndSalt = concatenateByteArrays(pass, salt);
            byte[] hash = new byte[0];
            byte[] keyAndIv = new byte[0];
            for (int i = 0; i < 3 && keyAndIv.length < 48; i++) {
                final byte[] hashData = concatenateByteArrays(hash, passAndSalt);
                MessageDigest md = null;
                md = MessageDigest.getInstance("MD5");
                hash = md.digest(hashData);
                keyAndIv = concatenateByteArrays(keyAndIv, hash);
            }
            final byte[] keyValue = Arrays.copyOfRange(keyAndIv, 0, 32);
            final SecretKeySpec key = new SecretKeySpec(keyValue, "AES");
            final byte[] iv = Arrays.copyOfRange(keyAndIv, 32, 48);
            final Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
            cipher.init(Cipher.DECRYPT_MODE, key, new IvParameterSpec(iv));
            final byte[] clear = cipher.doFinal(inBytes, 16, inBytes.length - 16);
            String contentDecoded = new String(clear, StandardCharsets.UTF_8);
            fos.write(contentDecoded.getBytes());
            fos.close();
            System.out.println("Decrypt is completed");
        }catch(Exception e){
            e.printStackTrace();
        }
    }

    public static byte[] concatenateByteArrays(byte[] a, byte[] b) {
        return ByteBuffer
                .allocate(a.length + b.length)
                .put(a).put(b)
                .array();
    }
}

This working implementation is certainly helpful for the OP. However, the decryption method contains unnecessary weaknesses of the code posted in the question: In contrast to the OP's encryption counterpart, the data is not processed in chunks and only UTF 8 decodable plaintexts can be decrypted. There are also resource leaks. — Topaco, Oct 21 '20 at 05:05
Thanks for the answer!! So if i want to do base64 encoding/decoding. then i should do base64 encoding instead of utf8 everywhere right?. (1)encode data to be writted to plaintext file(2)encode salt. (3) encode salt at decryption (4) decode data after decryption. But with this changes im getting the error: "javax.crypto.IllegalBlockSizeException: Input length must be multiple of 16 when decrypting with padded cipher" — Anees U, Oct 21 '20 at 05:52
@Topaco, If i do decryption by chunks im getting this error. Please suggest how to base64 encode/decode and do decoding by chunks: "javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption." >> CipherInputStream cis = new CipherInputStream(fis, cipher); int b; byte[] d = new byte[8]; while((b = cis.read(d)) != -1) { fos.write(d, 0, b); } fos.flush(); fos.close(); — Anees U, Oct 21 '20 at 06:01
@Anees U: can you kindly mark my answer as "accepted", thanks? Regarding your other question to Topaco - this is a complete other question and should be handled seperately as we are leaving the compatibility with OpenSSL-encryption. — Michael Fehr, Oct 21 '20 at 06:29
@AneesU - A decryption is tailored to its corresponding encryption. Currently you make _contradictory_ statements about encryption. On the one hand you reference [this code](https://stackoverflow.com/q/64423926) for encryption where no Base64 encoding is done (neither with nor without the first block containing the salt). On the other hand you are describing that the first block is not Base64 encoded, while the rest is (which by the way OpenSSL does not do either). Please clarify this in your question or better yet post the code that is used to encrypt. — Topaco, Oct 21 '20 at 07:19
@MichaelFehr,@Topaco, Thanks, my requirement is to encode with base64 ad decode with with base64. kindly suggest on my new question: https://stackoverflow.com/questions/45133268/java-decrypt-a-file-with-base64 — Anees U, Oct 21 '20 at 07:54

Decrypt aes encrypted file in java sha1 openssl

1 Answers1

Linked