4

I have this Python method on the server to encrypt a string into bytes (AES/CBC).

class AESCipher(object, key):
    def __init__(self, key): 
        self.bs = AES.block_size
        self.key = hashlib.sha256(key.encode()).digest()

    def encrypt(self, raw):
        raw = self._pad(raw)
        iv = Random.new().read(AES.block_size)
        cipher = AES.new(self.key, AES.MODE_CBC, iv)
        return base64.b64encode(iv + cipher.encrypt(raw.encode()))

    def decrypt(self, enc):
        enc = base64.b64decode(enc)
        iv = enc[:AES.block_size]
        cipher = AES.new(self.key, AES.MODE_CBC, iv)
        return self._unpad(cipher.decrypt(enc[AES.block_size:])).decode('utf-8')

    def _pad(self, s):
        return s + (self.bs - len(s) % self.bs) * chr(self.bs - len(s) % self.bs)

The output of encrypt() is in bytes like this: b'PMgMOkBkciIKfWy/DfntVMyAcKtVsM8LwEwnTYE5IXY='

I would like to store this into database, and send it as string via API to Kotlin. And in there I would like to decrypt it via the same shared secret key.

  1. In what format do I save the bytes above into database?
  2. Once arrived in Kotlin client, how do I convert that string into ByteArray?

My theory is that I have to store the bytes as base64 string in the database. And on the other side I have to decode the string as base64 into bytes. Is this approach correct? Will the encryption/decryption work like this end-to-end with the code below?

    fun decrypt(context:Context, dataToDecrypt: ByteArray): ByteArray {
            val cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING")
            val ivSpec = IvParameterSpec(getSavedInitializationVector(context))
            cipher.init(Cipher.DECRYPT_MODE, getSavedSecretKey(context), ivSpec)
            val cipherText = cipher.doFinal(dataToDecrypt)
            val sb = StringBuilder()
            for (b in cipherText) {
                sb.append(b.toChar())
            }   
            return cipherText
    }
    
    fun getSavedSecretKey(context: Context): SecretKey {
            val sharedPref = PreferenceManager.getDefaultSharedPreferences(context)
            val strSecretKey = sharedPref.getString("secret_key", "")
            val bytes = android.util.Base64.decode(strSecretKey, android.util.Base64.DEFAULT)
            val ois = ObjectInputStream(ByteArrayInputStream(bytes))
            val secretKey = ois.readObject() as SecretKey
            return secretKey
    }
    
    fun getSavedInitializationVector(context: Context) : ByteArray {
            val sharedPref = PreferenceManager.getDefaultSharedPreferences(context)
            val strInitializationVector = sharedPref.getString("initialization_vector", "")
            val bytes = android.util.Base64.decode(strInitializationVector, android.util.Base64.DEFAULT)
            val ois = ObjectInputStream(ByteArrayInputStream(bytes))
            val initializationVector = ois.readObject() as ByteArray
            return initializationVector
        }

UPDATE

I have tried to remove the Base64 to remove the memory overhead as suggested.

Python:

    def encrypt(self, raw):
        raw = self._pad(raw)
        iv = Random.new().read(AES.block_size)
        cipher = AES.new(self.key, AES.MODE_CBC, iv)
        return iv + cipher.encrypt(raw.encode())

So this is no longer possible.

enc = AESCipher('abc').encrypt("myLife")
value_to_save_in_db = enc.decode("utf8")

So I need to find a way to store the byte array directly in the database. I think I should be able to do this as blob. But some challenges remain as how to send the bytearray as part of JSON over the API to the android device. I think I have to convert it to Base64 string again. Not sure if I have gained anything in that case...

Houman
  • 64,245
  • 87
  • 278
  • 460
  • 1
    Why do you hash the key? The key was a password? – kelalaka Mar 28 '21 at 14:17
  • The Python code is problematic regarding authentication tag (is not automatically appended to the ciphertext by PyCryptodome), padding (GCM does not need padding) and nonce length (for GCM a 12 bytes nonce is recommended), see [PyCryptodome - GCM mode, incl. the Note section](https://pycryptodome.readthedocs.io/en/latest/src/cipher/modern.html#gcm-mode). The codes are incompatible e.g. because of the different handling of tag and IV. – Topaco Mar 28 '21 at 14:20
  • @Topaco Thank you for the points you have raised. I need to read more about GCM to get it right and have taken notes from what you said. In the meanwhile I have edited the question and replaced GCM with CBC. I think this version will now mitigate most of your concerns about authentication, nonce and tag. Would you please elaborate a bit more on IV and how I could change them to match? – Houman Mar 28 '21 at 14:47
  • In the Python code, IV and ciphertext are concatenated and Base64 encoded. In the Kotlin code the counterpart to this is missing, namely the Base64 decoding of the Python data and the subsequent separation of IV and ciphertext (the first 16 bytes are the IV, the rest the actual ciphertext). Instead, the IV is determined from the context and Base64 decoded (analogous to the key). So both codes do not match regarding the IV. Btw, PyCryptodome supports PKCS5/PKCS7 padding, s. [Crypto.Util.Padding](https://pycryptodome.readthedocs.io/en/latest/src/util/util.html#crypto-util-padding-module). – Topaco Mar 28 '21 at 15:50
  • I' ve posted a possible Kotlin implementation that decrypts the ciphertext of the Python code. – Topaco Mar 28 '21 at 16:38

1 Answers1

2

The following Kotlin code:

val decrypted = decrypt("blEOKMQtUbNOzJbvEkL2gNhjF+qQ/ZK84f2ADu8xyUFme6uBhNYqvEherF/RRO9YRImz5Y04/ll+T07kqv+ExQ==");
println(decrypted);

decrypts a ciphertext of the Python code. Here decrypt() is:

fun decrypt(dataToDecryptB64 : String) : String {

    // Base64 decode Python data
    val dataToDecrypt = Base64.getDecoder().decode(dataToDecryptB64)

    // Separate IV and Ciphertext
    val ivBytes = ByteArray(16)
    val cipherBytes = ByteArray(dataToDecrypt.size - ivBytes.size)
    System.arraycopy(dataToDecrypt, 0, ivBytes, 0, ivBytes.size)
    System.arraycopy(dataToDecrypt, ivBytes.size, cipherBytes, 0, cipherBytes.size)

    // Derive key
    val keyBytes = MessageDigest.getInstance("SHA256").digest("abc".toByteArray(Charsets.UTF_8))

    // Decrypt
    val cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING")
    cipher.init(Cipher.DECRYPT_MODE, SecretKeySpec(keyBytes, "AES"), IvParameterSpec(ivBytes))
    val cipherText = cipher.doFinal(cipherBytes)

    return String(cipherText, Charsets.ISO_8859_1)
}

For this, the ciphertext was generated using the posted Python class AESCipher as follows:

plaintext = 'The quick brown fox jumps over the lazy dog'
cipher = AESCipher('abc')
ciphertext = cipher.encrypt(plaintext)
print(ciphertext.decode('utf8')) # Base64 string, which can be stored e.g. in a DB

I applied the originally posted Python implementation that derives the key using SHA256. However, if the key is derived from a password, for security reasons not SHA256 but a reliable key derivation function, e.g. Argon2 or PBKDF2, should be used.

The Kotlin code first Base64 decodes the Python data and then separates IV and the actual ciphertext. Then, the key is derived by generating the SHA256 hash of the password. Finally the data is decrypted.


The current Python code Base64 encodes the data so that it can be stored as a string in the DB. Alternatively, the Python code could be modified so that no Base64 encoding is performed, and the raw data can be stored (which requires less memory, Base64 overhead: 33%).
Depending on the solution chosen, the Kotlin code may or may not need to Base64 decode the data.

Topaco
  • 40,594
  • 4
  • 35
  • 62
  • Thank you so much for your detailed answer. I'm still digesting it. But as you mentioned the key 'abc' has too few bytes to be accepted by AES in the first place. So my original entry of `self.key = hashlib.sha256(key.encode()).digest()` would rather be valid. This key was intended to be a shared-key that both Kotlin and Python have access to, otherwise they won't be able to encrypt/decrypt the information, correct? So is your suggestion to take a long string and encrypt it with Argon2 and use that hash as shared-key? – Houman Mar 28 '21 at 18:15
  • 1
    Symmetric encryption (e.g. AES) requires the same key on the encryption and decryption side. If this has to be exchanged over an insecure channel, an asymmetric algorithm can be used, e.g. RSA (see hybrid encryption). An AES key with sufficient entropy (e.g. a pseudo random key) can be applied directly. But if you want to use a password (with generally low entropy), you need a reliable [key derivation function](https://en.wikipedia.org/wiki/Key_derivation_function), e.g. Argon2, PBKDF2, etc., which derives the key from the password, s. also [here](https://security.stackexchange.com/a/16357). – Topaco Mar 28 '21 at 19:31
  • I understand what you mean. My plan was to store the shared-key securely via https://github.com/klaxit/hidden-secrets-gradle-plugin on Android. It encodes it separately as SHA256 and it can't be that easily discovered during disassembly operation, if someone tried to hack in. I know it's not a perfect solution. Thanks for the other suggestions. I have tested the whole code you provided here and I can't thank you enough. I don't think I could have solved this by myself. P.S. I have provided an update in the question regarding Base64, if you don't mind. – Houman Mar 28 '21 at 20:15
  • If the ciphertext in the Python code is not Base64 encoded into a string, then it is returned as a bytes like object. The Kotlin counterpart for this is a `ByteArray`. So it's not clear to me why the ciphertext should be a string. I would rather expect a `ByteArray` here, so that the problem would not arise in the first place. What generally is not allowed is to convert the ciphertext to a string with a charset encoding like UTF-8, because this would corrupt the data, see [here](https://stackoverflow.com/a/9098905/9014097). – Topaco Mar 28 '21 at 21:59
  • 1
    Apologies, it was late last night and I have made a silly mistake. I have updated the update section in my question. So yes, by removing the Base64, I'm no longer able to encode the result to UTF8 in order to store it in the database. I think I have to store the byte array as blob in the database. I'll give it a shot. – Houman Mar 29 '21 at 09:25