What algorithm to use when creating an encrypting log4j appender

Question

I've created a RollingFileAppender which encrypts the output to a log4j log file. Currently it uses AES/ECB/NoPadding, and it works fine.

Here is how we create the cipher

public static Cipher getCipher(boolean encrypt) throws Exception {
    //https://en.wikipedia.org/wiki/Stream_cipher       
    byte[] key = ("sometestkey").getBytes("UTF-8");
    MessageDigest sha = MessageDigest.getInstance("SHA-1");
    key = sha.digest(key);
    key = Arrays.copyOf(key, 16); // use only first 128 bit

    Key k = new SecretKeySpec(key,"AES");
    Cipher cipher = Cipher.getInstance("AES/ECB/NoPadding");
    if (encrypt) {
        cipher.init(Cipher.ENCRYPT_MODE, k);
    } else {
        cipher.init(Cipher.DECRYPT_MODE, k);
    }
    return cipher;
}

Here's how we create the appender :

public class EncryptingRollingFileAppender extends RollingFileAppender {
    private CipherOutputStream s;
    private Cipher cipher; 
    public EncryptingRollingFileAppender() {super();}
    public EncryptingRollingFileAppender(Layout layout, String filename, boolean append) throws IOException {super(layout, filename, append);}
    public EncryptingRollingFileAppender(Layout layout, String filename) throws IOException {super(layout, filename);}

    @Override
    protected OutputStreamWriter createWriter(OutputStream outputStream) {
        if (cipher==null) {
            try {
                cipher = DecryptionTools.getCipher(true);
                s = new CipherOutputStream(outputStream, cipher);
            } catch (Throwable t) {
                throw new RuntimeException("failed to initialise encrypting file appender",t);
            }
        }
        OutputStreamWriter out = super.createWriter(s);
        return out;
    }   
}

We can decrypt the file by using

getCipher(false)

to create an appropriate decryption stream.

The issue is that our security team are haggling about key management. They don't like the use symetric key encryption, and would prefer us to use a key pair rather than a simple password that we have to manage in some way.

Does anyone know of a non-padding ECB encryption technique that would use a key pair, and would be appropriate for this kind of stream encryption and decryption?

Answer 1

The comments suggesting using hybrid encryption and PGP are right.
PGP is the defacto standard for hybrid encryption of files, and it is a much more robust solution that ECB mode AES.

Due to the nature of PGP it is going to work slightly differently to your existing solution.

PGP messages have a header and a footer, so you will want each file to be individually encrypted (you wont just be able to decrypt individual blocks like you can with plain ECB mode encryption).

It looks like you are using log4j 1.2, I have created a working implementation of a PGP encrypting RollingFileAppender.

To use the example, generate an armour encoded PGP public key, run the main class then decrypt the file with any PGP tool (I used GnuPG for creating the keys and decrypting).

The example was built against log4j:log4j:1.2.17 and org.bouncycastle:bcpg-jdk15on:1.56

Answer 2

You are pretty close to what you want to achieve. Using Log4j 1.2 (because you cannot subclass RollingFileAppender directly in Log4j 2) you can generate a password on the fly for each log file, encrypt the password with RSA and store next to it. Then use the password to produce an AES CipherOutputStream for the log appender.

public class EncryptingRollingFileAppender extends RollingFileAppender {

    private CipherOutputStream s;

    private Cipher cipher;

    private byte[] secretKey;

    public EncryptingRollingFileAppender(Layout layout, String filename, boolean append) throws IOException {
        super(layout, filename, append);
        writeKeyFile(filename);
    }

    public EncryptingRollingFileAppender(Layout layout, String filename) throws IOException {
        super(layout, filename);
        writeKeyFile(filename);
    }

    private void writeKeyFile(final String logfilename) throws IOException {
        final int dot = logfilename.lastIndexOf('.');
        final String keyfilename = (dot == -1 ? logfilename : logfilename.substring(0, dot)) + ".key";
        try (FileOutputStream out = new FileOutputStream(keyfilename)) {
            out.write(DecryptionTools.encryptPasswordBase64(secretKey).getBytes(ISO_8859_1));
        } catch (InvalidKeyException | NoSuchAlgorithmException | NoSuchPaddingException | KeyStoreException
                | CertificateException e) {
        }
    }

    @Override
    protected OutputStreamWriter createWriter(OutputStream outputStream) {
        System.out.println("createWriter()");
        if (cipher == null) {
            secretKey = DecryptionTools.generateRandomKey(16).getBytes(ISO_8859_1);
            try {
                cipher = DecryptionTools.getCipher(true, secretKey);
            } catch (InvalidKeyException e) {
                System.out.println("InvalidKeyException");
            }
            s = new CipherOutputStream(outputStream, cipher);
        }

        OutputStreamWriter out = super.createWriter(s);
        return out;
    }
}

You´ll need several helper functions for reading the private key from a file or from a Java key store which can be found here.

The test file TestEncryptingRollingFileAppender shows how to write an encrypted log and read it back.

import static com.acme.DecryptionTools.getCipher;
import static com.acme.DecryptionTools.decryptPasswordBase64;

public class TestEncryptingRollingFileAppender {

    @SuppressWarnings("deprecation")
    @Test
    public void testAppender() throws IOException, InvalidKeyException, NoSuchAlgorithmException, NoSuchPaddingException, KeyStoreException, CertificateException, InvalidKeySpecException, IllegalBlockSizeException, BadPaddingException {

        final File logfile = File.createTempFile("testlog_", ".log");
        final String logfilename = logfile.getAbsolutePath();

        final Logger lggr = LogManager.getLogger(TestEncryptingRollingFileAppender.class);
        final EncryptingRollingFileAppender appender = new EncryptingRollingFileAppender(new SimpleLayout(), logfilename, true);

        appender.append(new LoggingEvent(lggr.getClass().getName(), lggr, Priority.INFO, "Test Log Line #1", null));
        appender.append(new LoggingEvent(lggr.getClass().getName(), lggr, Priority.INFO, "Test Log Line #1", null));

        final int dot = logfilename.lastIndexOf('.');
        byte[] key = decryptPasswordBase64(new String(Files.readAllBytes(Paths.get(logfilename.substring(0, dot)+".key"))));

        StringBuilder logContent = new StringBuilder();
        try (FileInputStream instrm = new FileInputStream(logfilename);
             CipherInputStream cistrm = new CipherInputStream(instrm, getCipher(false, key))) {
            int c;
            while ((c=cistrm.read())!=-1)
                logContent.append((char) c);
        }

        assertEquals("INFO - Test Log Line #1\r\nINFO - Test Log Line #1", logContent.toString());

    }
}