1

C# code uses AES to encrypt arrays of bytes.

I have written a Python program using PyCryptodome to do the same thing, but the encrypted bytes are always different from the result when I use the C# code, and I made sure to:

  • set the IV to the same value in both (just for test purpose)
  • ensure the key is the same in both
  • ensure the raw data is the same

What I am encrypting: an array of bytes. The bytes represent primarily TLD-format data.

The Python program will be a part of a utility that will generate streams on the fly that will be processed by a Web application written in C#.

Using http://aes.online-domain-tools.com, I actually could decrypt the bytes produced by the C# code and verify that it's using AES, and that the raw data is correct.

The question is: what else could be the discriminating factor?

Python snippets:

      from Crypto.Cipher import AES
      from Crypto import Random
      from Crypto.Util.Padding import pad

**** Correction ***
        aes_cipher = AES.new(bytes(key, 'UTF-8'), AES.MODE_CBC)
#
# Correct, the above call would use a random IV value.
# For debugging & learning purpose, I halted this in the 
# debugger and manually set 
# aes_cipher.IV = <a given value>
# and used the same IV in the C# code to try and keep all known inputs identical.
# 
        aes_cipher.block_size = 128
        aes_cipher.key_size = 128   # bits
        encrypted_pack = aes_cipher.encrypt(pad(pack, 16))

        # Tack on to the beginning the 16 bytes of the "IV"

        # FYI - the C# decryption function strips off the first 16 IV bytes
        encrypted_pack = aes_cipher.IV + encrypted_pack

        return encrypted_pack

C# snippets

            AesCipher = new RijndaelManaged();
            AesCipher.KeySize = 128;  // 192, 256

            // BlockSize: 128-bit == 16 bytes. 
            // 128-bit is the default for RijndaelManaged
            AesCipher.BlockSize = 128;

            AesCipher.Mode = CipherMode.CBC;
            AesCipher.Padding = PaddingMode.Zeros;

...
...

# 
# Yes, GenerateIV() generates a random IV.
# As mentioned above, I overrode this by setting 
# AesCipher.IV = <the same value as above>
# 

                AesCipher.GenerateIV();                
                setKey(key);   // converts a string of decimal digits to string of hex digits  

                ICryptoTransform transform = AesCipher.CreateEncryptor();
                byte[] encrypted = transform.TransformFinalBlock(buf, 0, buf.Length);
                byte[] result = new byte[encrypted.Length + 16];

                Buffer.BlockCopy(AesCipher.IV, 0, result, 0, 16);
                Buffer.BlockCopy(encrypted, 0, result, 16, encrypted.Length);

                return result;


***
Update
***
There was another problem that I just discovered and fixed.
The key was being saved as a 32-byte rather than 16-byte bytearray, which would explain the gigantic discrepancy from online tool results.

Solved easiy with 
```byte_key = binascii.unhexlify(key)
Once I did that, the returned by both pieces of code matched, and they matched what was in the online tool, too.
Sneaky because in the debugger, it's easy to miss because the values look the same.
Guy
  • 666
  • 1
  • 10
  • 34
  • `AesCipher.GenerateIV()` is generating a random IV, if I remember correctly. This is different from _set the IV to the same value in both (just for test purpose)_ – xanatos Dec 27 '20 at 14:06
  • I'm not really sure what `aes_cipher.encrypt(pad(pack, 16))`, but it almost sounds like it's padding the data and then encrypting it. I'm not a Python programmer so I could be wrong, but that's just how it sounds. Your C# code, on the other hand, is certainly encrypting the data and then copying it to the offset position in the resulting array. – ProgrammingLlama Dec 27 '20 at 14:09
  • Contrary to the description in the question, a random IV is also used in the Python code, since no IV is passed in the 3rd parameter when instantiating the AES object, s. the [documentation](https://pycryptodome.readthedocs.io/en/latest/src/cipher/classic.html?highlight=CBC%20cipher%20object#cbc-mode). – Topaco Dec 27 '20 at 14:30
  • @John - I know that if I don't pad the input, then I will get an exception "Data must be padded to 128 byte boundary in CBC mode" – Guy Dec 27 '20 at 15:25
  • 1
    Your key is likely also decoded differently in the snippets, but you don't show that code here. Check that again. Also, setting the key size should not be done if you explicitly pass a key in. That key will be used. If you set the key size then there is a small chance that the library misbehaves and generates a new key for you. – Artjom B. Dec 27 '20 at 16:48

1 Answers1

2

AesCipher.GenerateIV() is generating a random IV, if I remember correctly. This is different from

set the IV to the same value in both (just for test purpose)

And the default padding of Crypto.Util.Padding.pad is

style (string) – Padding algorithm. It can be ‘pkcs7’ (default), ‘iso7816’ or ‘x923’.

that is different from:

AesCipher.Padding = PaddingMode.Zeros;

Fully working C# and Python samples:

public static byte[] SimpleEncryptAesVariableLengthCbcZeros(string key, byte[] iv, byte[] plain)
{
    byte[] key2 = Encoding.UTF8.GetBytes(key);

    if (key.Length == 0 || key.Length > 32)
    {
        throw new ApplicationException("Illegal length for key");
    }

    int keySize = key2.Length <= 16 ? 128 : key2.Length <= 24 ? 192 : 256;

    using (var aesCipher = new RijndaelManaged())
    {
        aesCipher.KeySize = keySize;

        // BlockSize: 128-bit == 16 bytes. 
        // 128-bit is the default for RijndaelManaged
        aesCipher.BlockSize = 128;

        aesCipher.Mode = CipherMode.CBC;
        aesCipher.Padding = PaddingMode.Zeros;

        if (iv == null)
        {
            // IV as calculated by http://aes.online-domain-tools.com/
            // SHA1(key) truncated to 16 bytes
            iv = SHA1.HashData(key2);
            Array.Resize(ref iv, aesCipher.BlockSize / 8);
        }
        else if (iv.Length != aesCipher.BlockSize / 8)
        {
            throw new ApplicationException("Illegal length for IV");
        }

        aesCipher.IV = iv;

        // Key is padded with bytes set to 0
        Array.Resize(ref key2, aesCipher.KeySize / 8);
        aesCipher.Key = key2;

        using (var encryptor = aesCipher.CreateEncryptor())
        {
            var encrypted = encryptor.TransformFinalBlock(plain, 0, plain.Length);

            var iv_encrypted = new byte[iv.Length + encrypted.Length];
            Array.Copy(iv, 0, iv_encrypted, 0, iv.Length);
            Array.Copy(encrypted, 0, iv_encrypted, iv.Length, encrypted.Length);
            return iv_encrypted;
        }
    }
}

// https://stackoverflow.com/a/311179/613130
public static byte[] StringToByteArray(string hex)
{
    hex = hex.Replace(" ", string.Empty);

    byte[] bytes = new byte[hex.Length / 2];

    for (int i = 0; i < hex.Length; i += 2)
    {
        bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16);
    }

    return bytes;
}

public static string ByteArrayToString(byte[] bytes, string join)
{
    string res = string.Join(join, Array.ConvertAll(bytes, x => x.ToString("x2")));
    return res;
}


static void Main(string[] args)
{
    string key = "abcdefghabcdefghabcdefghabcdefgh";
    byte[] plain = StringToByteArray("0000000000000000000000000000000001");
    var res = SimpleEncryptAesVariableLengthCbcZeros(key, null, plain);
    var res2 = ByteArrayToString(res, " ");
    Console.WriteLine(res2);
}

and (note that is probably the second or third time I write Python in my life, so I'm not very sure of its quality, and surely as hell it isn't optimized):

from Crypto.Cipher import AES
from Crypto import Random
import hashlib 
#from Crypto.Util.Padding import pad

#key must be string
#iv must be bytes or None
#plain must be bytes
def SimpleEncryptAesVariableLengthCbcZeros(key, iv, plain):
    key2 = bytes(key, 'UTF-8')
    
    if len(key2) == 0 or len(key2) > 32:
        raise Exception('Illegal length for key')

    keySize = 128 if len(key2) <= 16 else 192 if len(key2) <= 24 else 256

    if iv == None:
        #IV as calculated by http://aes.online-domain-tools.com/
        #SHA1(key) truncated to 16 bytes
        h = hashlib.sha1()
        h.update(key2)

        iv = h.digest()
        iv = iv[0:16]
    elif len(iv) != 128 // 8:
        raise Exception('Illegal length for iv')

    #Key is padded with bytes set to 0
    key2 = key2 + b'\0' * (keySize // 8 - len(key2))

    aes_cipher = AES.new(key2, AES.MODE_CBC, iv)

    aes_cipher.key_size = keySize
    aes_cipher.block_size = 128

    padded = plain
    
    #zero padding
    if len(padded) % 16 != 0:
        padded = padded + b'\0' * (16 - len(padded) % 16)

    encrypted = aes_cipher.encrypt(bytes(padded))

    iv_encrypted = iv + encrypted

    return iv_encrypted

key = 'abcdefghabcdefghabcdefghabcdefgh'
plain = bytearray.fromhex('0000000000000000000000000000000001')
iv_encrypted = SimpleEncryptAesVariableLengthCbcZeros(key, None, plain)
print(' '.join(["{:02x}".format(x) for x in iv_encrypted]))
xanatos
  • 109,618
  • 12
  • 197
  • 280
  • You are correct that that generates a random IV. – ProgrammingLlama Dec 27 '20 at 14:14
  • I'll add a clarification in the question that even if I step through the code and override the IV with a specific value (same as used in the C# code), the end-results differ. – Guy Dec 27 '20 at 14:55
  • @Guy have you fixed the padding? To check if it is the padding, try encrypting a block of at least 32 bytes. What are you encrypting? Text? If yes, are encrypting text encoded in the same way? (UTF8 or Unicode) – xanatos Dec 27 '20 at 15:00
  • @xanatos - can you please explain what you mean by your suggestion? – Guy Dec 27 '20 at 15:10
  • @Guy `AesCipher.Padding = PaddingMode.PKCS7` – xanatos Dec 27 '20 at 15:12
  • @xanatos - I just tried this change to see if the results would be equal, but they still are not! I also discovered that the TransformFinalBlock essentially does the padding for you. My particular in-arg was 66 bytes, and the output was 80 bytes. – Guy Dec 27 '20 at 16:02
  • @xanatos - I think you're on to something with the padding. I noticed now that the pad characters in PKCS7 (Python-land) are \x0e because there are 14 (0e) characters to fill (80-66). The C# decryptor is using 0-padding. – Guy Dec 27 '20 at 17:42