-1

I have information security project about encrypting file using AES. and the using key in this algorithm is also encrypted using RSA algorithm and public key, the problem is: after encrypting the random key it returns array byte[], how this array byte converted into key so I can encrypt the file? NOTE [public_Key is generated from user using JPasswordField and this is the challenge I faced from my course project]

public void AESEncryption(File file) throws FileNotFoundException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException {
    String data;
    SecretKey random_key;
    int key_size=128;

    Scanner myReader = new Scanner(file);
    while (myReader.hasNextLine()) {
        data = myReader.nextLine();
    }

    // create GenerateKey object to access public key
    // GenerateKey is my personal class and contain public key
    GenerateKey key = new GenerateKey();

    // convert public key to string
    String public_Key = key.PublicKey.getText();

    // convert string public key to secret key
    byte[] decodedKey = Base64.getDecoder().decode(public_Key);
    SecretKey originalKey = new SecretKeySpec(decodedKey, 0, decodedKey.length, "AES");

    // generate random key
    KeyGenerator g = KeyGenerator.getInstance("AES");
    // give it size
    g.init(key_size);
    random_key = g.generateKey();

    // encrypt the random key with RSA and public key
    byte[] random_byteKey = random_key.getEncoded();
    Cipher cipher_Key = Cipher.getInstance("RSA/ECB/PKCS1Padding");
    cipher_Key.init(Cipher.ENCRYPT_MODE, originalKey);
    byte[] encrypted_key = cipher_Key.doFinal(random_byteKey); //RSA key

    // after generating RSA key we will Encrypt file using RSA key
    byte[] byte_message = data.getBytes();
    Cipher cipherTxt = Cipher.getInstance("AES/GCM/NoPadding");
    // the problem in here 
    cipherTxt.init(Cipher.ENCRYPT_MODE, encrypted_key);

    byte[] encByte = cipherTxt.doFinal(byte_message);
}
kandy
  • 1
  • 1
  • 2
    Please do not begin variable names with an upper case letter! It confuses the reader as it is not immediately clear whether you referenced a class or a variable in your statements. – Janez Kuhar Apr 17 '22 at 18:51
  • sorry for confusing you, I have edited it. – kandy Apr 17 '22 at 19:20

1 Answers1

1

You are not understanding what you need to do. First you generate a random AES key that is used solely for the data encryption. Then you encrypt that key with RSA using the trusted RSA public key which is part of the key pair of the receiver. So you never have to convert either the public key or the RSA ciphertext to a symmetric key.

As an aside, instead of using Cipher#doFinal() you should use Cipher#wrap() , which takes a symmetric key. That way you don't have to encode them to a byte array. It may also be more secure if a hardware module is used, for instance, depending on the Cipher implementation.

I'd strongly suggest you generate separate methods for these separate steps as well as for the file handling.

In the end, you'll need something more akin to this:

public static void hybridEncrypt(RSAPublicKey publicKey, File in, File out) throws IOException, InvalidKeyException {
    int key_size=128;
    
    try {
        KeyGenerator g = KeyGenerator.getInstance("AES");
        g.init(key_size);
        SecretKey dataKey = g.generateKey();

        // encrypt the random data key with the RSA public key
        Cipher cipher_Key = Cipher.getInstance("RSA/ECB/PKCS1Padding");
        cipher_Key.init(Cipher.WRAP_MODE, publicKey);
        byte[] encryptedKey = cipher_Key.wrap(dataKey);

        Cipher cipherTxt = Cipher.getInstance("AES/GCM/NoPadding");
        cipherTxt.init(Cipher.ENCRYPT_MODE, dataKey);

        byte[] message = Files.readAllBytes(in.toPath());
        byte[] encryptedMessage = cipherTxt.doFinal(message);
        out.createNewFile();
        Files.write(out.toPath(), encryptedKey);
        Files.write(out.toPath(), encryptedMessage, StandardOpenOption.APPEND);
    } catch(NoSuchAlgorithmException | NoSuchPaddingException | IllegalBlockSizeException e) {
        throw new RuntimeException("RSA or AES/GCM not available", e);
    } catch (BadPaddingException e) {
        throw new RuntimeException("Padding failed for NoPadding", e);
    }
}

public static void main(String[] args) throws Exception {
    KeyPairGenerator kpGen = KeyPairGenerator.getInstance("RSA");
    kpGen.initialize(3072);
    KeyPair keyPairReceiver = kpGen.generateKeyPair();
    RSAPublicKey publicKeyReceiver = (RSAPublicKey) keyPairReceiver.getPublic();
    hybridEncrypt(publicKeyReceiver, new File("plain.txt"), new File("bla.bin"));
}

Beware that this is still not best practice code, for instance it uses the old PKCS#1 encryption instead of OAEP. Don't copy paste this guys - with encryption you need to understand what you are doing, and preferably use a well vetted high level library.

Maarten Bodewes
  • 90,524
  • 13
  • 150
  • 263
  • thank you very much for helping me!, sorry to tell you that I am not security information expert and I am still learning from my errors, I forget to explain that public_key basically generated from user using JPasswordField. – kandy Apr 17 '22 at 23:45
  • Remember, I don't know your scheme nor do I get what you are trying to do. I've just explained how a hybrid cryptosystem works, including code. If you're trying something "different' then you need to make a textual description of your protocol, including what you are trying to protect against and how. – Maarten Bodewes Apr 18 '22 at 00:50