ASP.NET AES Padding is invalid and cannot be removed

Question

This is a common problem but the solutions suggested here and here don't work for me, also they don't involve a database.

I'd like the code sample 'Failing code (WITH database)' to work.

This is my encrypt/decrypt code (copied from here: https://msdn.microsoft.com/en-us/library/system.security.cryptography.aes.aspx?cs-save-lang=1&cs-lang=vb&f=255&MSPPError=-2147217396#code-snippet-2):

Shared Function EncryptStringToBytes_Aes(ByVal plainText As String, ByVal Key() As Byte, ByVal IV() As Byte) As Byte()
    ' Check arguments.
    If plainText Is Nothing OrElse plainText.Length <= 0 Then
        Throw New ArgumentNullException("plainText")
    End If
    If Key Is Nothing OrElse Key.Length <= 0 Then
        Throw New ArgumentNullException("Key")
    End If
    If IV Is Nothing OrElse IV.Length <= 0 Then
        Throw New ArgumentNullException("Key")
    End If
    Dim encrypted() As Byte
    ' Create an Aes object
    ' with the specified key and IV.
    Using aesAlg As Aes = Aes.Create()

        aesAlg.Key = Key
        aesAlg.IV = IV

        ' Create a decrytor to perform the stream transform.
        Dim encryptor As ICryptoTransform = aesAlg.CreateEncryptor(aesAlg.Key, aesAlg.IV)
        ' Create the streams used for encryption.
        Using msEncrypt As New MemoryStream()
            Using csEncrypt As New CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write)
                Using swEncrypt As New StreamWriter(csEncrypt)

                    'Write all data to the stream.
                    swEncrypt.Write(plainText)
                End Using
                encrypted = msEncrypt.ToArray()
            End Using
        End Using
    End Using

    ' Return the encrypted bytes from the memory stream.
    Return encrypted

End Function 'EncryptStringToBytes_Aes


Shared Function DecryptStringFromBytes_Aes(ByVal cipherText() As Byte, ByVal Key() As Byte, ByVal IV() As Byte) As String
    ' Check arguments.
    If cipherText Is Nothing OrElse cipherText.Length <= 0 Then
        Throw New ArgumentNullException("cipherText")
    End If
    If Key Is Nothing OrElse Key.Length <= 0 Then
        Throw New ArgumentNullException("Key")
    End If
    If IV Is Nothing OrElse IV.Length <= 0 Then
        Throw New ArgumentNullException("Key")
    End If
    ' Declare the string used to hold
    ' the decrypted text.
    Dim plaintext As String = Nothing

    ' Create an Aes object
    ' with the specified key and IV.
    Using aesAlg As Aes = Aes.Create()
        aesAlg.Key = Key
        aesAlg.IV = IV

        ' Create a decrytor to perform the stream transform.
        Dim decryptor As ICryptoTransform = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV)

        ' Create the streams used for decryption.
        Using msDecrypt As New MemoryStream(cipherText)

            Using csDecrypt As New CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read)

                Using srDecrypt As New StreamReader(csDecrypt)
                    ' Read the decrypted bytes from the decrypting stream
                    ' and place them in a string.
                    plaintext = srDecrypt.ReadToEnd()
                End Using   <= PADDING ERROR THROWN HERE
            End Using

            'cipherText = msDecrypt.ToArray() 'added by me

        End Using
    End Using

    Return plaintext

End Function 'DecryptStringFromBytes_Aes 

Working sample (without database)

    Dim original As String = "Here is some data to encrypt!"

    Dim key As Rfc2898DeriveBytes = New Rfc2898DeriveBytes(_sharedSecret, _salt)

    Try

        ' Create a new instance of the Aes
        ' class.  This generates a new key and initialization 
        ' vector (IV).
        Using myAes As Aes = Aes.Create()

            myAes.Key = key.GetBytes(myAes.KeySize / 8)
            myAes.IV = key.GetBytes(myAes.BlockSize / 8)

            ' Encrypt the string to an array of bytes.
            Dim encrypted As Byte() = EncryptStringToBytes_Aes(original, myAes.Key, myAes.IV)

            ' Decrypt the bytes to a string.
            Dim roundtrip As String = DecryptStringFromBytes_Aes(encrypted, myAes.Key, myAes.IV)

            'Display the original data and the decrypted data.
            ltStatus.Text = String.Format("Original:   {0}", original)
            ltStatus.Text += String.Format("Round Trip: {0}", roundtrip)
        End Using
    Catch ex As Exception
        Console.WriteLine("Error: {0}", ex.Message)
    End Try



    

Failing code (WITH database)

    Dim _salt As Byte() = Encoding.ASCII.GetBytes("o6806642kbM7c5")
    Dim _sharedSecret As String = "abcd"
    Dim original As String = "Here is some data to encrypt!"
    Dim key As Rfc2898DeriveBytes = New Rfc2898DeriveBytes(_sharedSecret, _salt)
    Dim encrypted As Byte()
    Try
        Using myAes As Aes = Aes.Create()
            myAes.Key = key.GetBytes(myAes.KeySize / 8)
            myAes.IV = key.GetBytes(myAes.BlockSize / 8)
            myAes.Padding = PaddingMode.PKCS7
            encrypted = EncryptStringToBytes_Aes(original, myAes.Key, myAes.IV)
        End Using
    Catch ex As Exception
        Console.WriteLine("Error: {0}", ex.Message)
    End Try
    'save to DB
    Dim myConnection As SqlConnection = GetConnection()
    Dim cmd As New SqlCommand("UPDATE banks set bank_name=@bankname WHERE id=1", myConnection)
    cmd.Parameters.Add(New SqlParameter("@bankname", encrypted))
    Try
        myConnection.Open()
        cmd.ExecuteScalar()
    Catch ex As Exception
        GlobalFunctions.LogError("banks:INSERT encrypted", ex.Message, LogLevel.Normal)
    Finally
        myConnection.Close()
    End Try

    'retreive from db
    Dim decrypted As String = ""
    myConnection = GetConnection()
    cmd = New SqlCommand("SELECT bank_name FROM banks where id=1", myConnection)
    Dim reader As SqlDataReader
    Try
        myConnection.Open()
        reader = cmd.ExecuteReader
        If reader.Read Then
            Using myAes As Aes = Aes.Create()
                myAes.Key = key.GetBytes(myAes.KeySize / 8)
                myAes.IV = key.GetBytes(myAes.BlockSize / 8)
                myAes.Padding = PaddingMode.PKCS7
                decrypted = DecryptStringFromBytes_Aes(reader("bank_name"), myAes.Key, myAes.IV)
            End Using
        Else
            GlobalFunctions.LogError("banks:nothing to be read?!?", LogLevel.Normal)
        End If

    Catch ex As Exception
        GlobalFunctions.LogError("banks:SELECT encrypted.", ex.Message, LogLevel.Normal)
    Finally
        myConnection.Close()
    End Try



    

But something goes wrong here:

A binary value is succesfully added in my MSSQL database in field bank_name of type varbinary(MAX). (I also tried smaller fields, e.g. varbinary(50))

But when I try to decrypt this field after retreival from the database I get the error Padding is invalid and cannot be removed. See the codeline with the comment '<= PADDING ERROR THROWN HERE' in the above code 'Failing code (WITH database)'.

I checked here and here. And I'm not passing an empty string Also I tried adding cipherText = msDecrypt.ToArray(), but the error already occurs before this line is hit.

UPDATE 2

My dumped values are:

ReportError stores the values in a textfield in the DB, the reported values are:

myAes.Key in
00000000 95 0C 95 EA 1D 40 0C FB 1D 3F B7 FB 73 FB 3F EA ���������������� 00000010 40 62 51 62 51 EA 62 73 B7 2E 1D C8 1D 51 51 95 ����������������

myAes.IV in
00000000 51 A6 84 73 95 C8 2E 62 84 C8 0C 62 C8 2E 1D 84 ����������������

encrypted in
00000000 FB FB B7 73 D9 51 A6 2E 95 73 62 73 3F 84 A6 40 ���������������� 00000010 B7 62 84 2E 51 95 EA 1D 51 A6 EA 2E 51 A6 51 95 ����������������

myAes.Key out
00000000 51 1D 73 40 EA A6 73 EA FB 73 73 A6 0C A6 D9 1D ���������������� 00000010 2E 3F FB 2E 73 A6 A6 0C A6 C8 95 0C D9 1D B7 73 ����������������

myAes.IV out
00000000 B7 95 51 73 B7 D9 95 EA 0C C8 95 95 0C 84 40 62 ����������������

encrypted out
00000000 FB FB B7 73 D9 51 A6 2E 95 73 62 73 3F 84 A6 40 ���������������� 00000010 B7 62 84 2E 51 95 EA 1D 51 A6 EA 2E 51 A6 51 95 ����������������

banks:SELECT encrypted. Padding is invalid and cannot be removed.

Here's the full code I use now:

Imports System.Security.Cryptography
Imports System.Data.SqlClient
Imports System.Text

Namespace HexDump
    Class Utils
        Public Shared Function HexDump(bytes As Byte(), Optional bytesPerLine As Integer = 16) As String
            If bytes Is Nothing Then
                Return "<null>"
            End If
            Dim bytesLength As Integer = bytes.Length

            Dim HexChars As Char() = "0123456789ABCDEF".ToCharArray()

            ' 8 characters for the address
            Dim firstHexColumn As Integer = 8 + 3
            ' 3 spaces
            ' - 2 digit for the hexadecimal value and 1 space
            ' - 1 extra space every 8 characters from the 9th
            Dim firstCharColumn As Integer = firstHexColumn + bytesPerLine * 3 + (bytesPerLine - 1) / 8 + 2
            ' 2 spaces 
            ' - characters to show the ascii value
            Dim lineLength As Integer = firstCharColumn + bytesPerLine + Environment.NewLine.Length
            ' Carriage return and line feed (should normally be 2)
            Dim line As Char() = (New [String](" "c, lineLength - Environment.NewLine.Length) + Environment.NewLine).ToCharArray()
            Dim expectedLines As Integer = (bytesLength + bytesPerLine - 1) / bytesPerLine
            Dim result As New StringBuilder(expectedLines * lineLength)

            Dim i As Integer = 0
            While i < bytesLength
                line(0) = HexChars((i >> 28) And &HF)
                line(1) = HexChars((i >> 24) And &HF)
                line(2) = HexChars((i >> 20) And &HF)
                line(3) = HexChars((i >> 16) And &HF)
                line(4) = HexChars((i >> 12) And &HF)
                line(5) = HexChars((i >> 8) And &HF)
                line(6) = HexChars((i >> 4) And &HF)
                line(7) = HexChars((i >> 0) And &HF)

                Dim hexColumn As Integer = firstHexColumn
                Dim charColumn As Integer = firstCharColumn

                For j As Integer = 0 To bytesPerLine - 1
                    If j > 0 AndAlso (j And 7) = 0 Then
                        hexColumn += 1
                    End If
                    If i + j >= bytesLength Then
                        line(hexColumn) = " "c
                        line(hexColumn + 1) = " "c
                        line(charColumn) = " "c
                    Else
                        'Dim b As Byte = bytes(i + j)
                        'line(hexColumn) = HexChars((b >> 4) And &HF)
                        'line(hexColumn + 1) = HexChars(b And &HF)
                        'line(charColumn) = (If(b < 32, "·"c, CChar(b)))
                        Dim b As Byte = bytes((i + j))
                        line(hexColumn) = HexChars(((b + 4) _
             And 15))
                        line((hexColumn + 1)) = HexChars((b And 15))
                        line(charColumn) = Microsoft.VisualBasic.ChrW(65533)

                    End If
                    hexColumn += 3
                    charColumn += 1
                Next
                result.Append(line)
                i += bytesPerLine
            End While
            Return result.ToString()
        End Function
    End Class
End Namespace




Public Class banks_financial
    Inherits System.Web.UI.Page

    Private _lang As String
    Private _registryId As Integer

    Private _salt As Byte() = Encoding.ASCII.GetBytes("o6806642kbM7c5")
    Private _sharedSecret As String = "abcd"

    Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
        Dim original As String = "Here is some data to encrypt!"
        Dim key As Rfc2898DeriveBytes = New Rfc2898DeriveBytes(_sharedSecret, _salt)
        Dim encrypted As Byte()
        Try
            Using myAes As Aes = Aes.Create()
                myAes.Key = key.GetBytes(myAes.KeySize / 8)
                myAes.IV = key.GetBytes(myAes.BlockSize / 8)
                myAes.Padding = PaddingMode.PKCS7
                encrypted = EncryptStringToBytes_Aes(original, myAes.Key, myAes.IV)

                ReportError("myAes.Key in", HexDump.Utils.HexDump(myAes.Key))
                ReportError("myAes.IV in", HexDump.Utils.HexDump(myAes.IV))
                ReportError("encrypted in", HexDump.Utils.HexDump(encrypted))

            End Using
        Catch ex As Exception
            Console.WriteLine("Error: {0}", ex.Message)
        End Try
        'save to DB
        Dim myConnection As SqlConnection = GetConnection()
        Dim cmd As New SqlCommand("UPDATE banks set bank_name=@bankname WHERE id=1", myConnection)
        cmd.Parameters.Add(New SqlParameter("@bankname", encrypted))
        Try
            myConnection.Open()
            cmd.ExecuteScalar()
        Catch ex As Exception
            GlobalFunctions.ReportError("banks:INSERT encrypted", ex.Message, LogLevel.Normal)
        Finally
            myConnection.Close()
        End Try

        'retreive from db
        Dim decrypted As String = ""
        myConnection = GetConnection()
        cmd = New SqlCommand("SELECT bank_name FROM banks where id=1", myConnection)
        Dim reader As SqlDataReader
        Try
            myConnection.Open()
            reader = cmd.ExecuteReader
            If reader.Read Then
                Using myAes As Aes = Aes.Create()
                    myAes.Key = key.GetBytes(myAes.KeySize / 8)
                    myAes.IV = key.GetBytes(myAes.BlockSize / 8)
                    myAes.Padding = PaddingMode.PKCS7

                    ReportError("myAes.Key out", HexDump.Utils.HexDump(myAes.Key))
                    ReportError("myAes.IV out", HexDump.Utils.HexDump(myAes.IV))
                    ReportError("encrypted out", HexDump.Utils.HexDump(reader("bank_name")))
                    decrypted = DecryptStringFromBytes_Aes(reader("bank_name"), myAes.Key, myAes.IV)
                    ReportError("decrypted", decrypted)
                End Using
            Else
                GlobalFunctions.ReportError("banks:nothing to be read?!?", LogLevel.Normal)
            End If

        Catch ex As Exception
            GlobalFunctions.ReportError("banks:SELECT encrypted.", ex.Message, LogLevel.Normal)
        Finally
            myConnection.Close()
        End Try

        ltStatus.Text = GetMessageStatus(decrypted, MsgType.ok)

   



End Class

Answer 1

The immediate problem is that your test code is misusing Rfc2898DeriveBytes/ PBKDF. It has nothing to do with the database. Each time you call GetBytes() on a given instance, it returns a different set of bytes (by design!). This is apparent with a bit of code like this:

    Dim salt = CryptoTools.GetRandomBytes(16)
    Dim PBKDF = New Rfc2898DeriveBytes("secret", salt, 10000)

    Dim a = PBKDF.GetBytes(32)
    Dim b = PBKDF.GetBytes(32)
    Dim c = PBKDF.GetBytes(32)

    If a.SequenceEqual(b) = False Then
        Console.WriteLine("A != B")
    End If
    ...

It's very clear with Intellisense:

The arrays are not at all alike. From the Remarks section on MSDN for GetBytes:

The Rfc2898DeriveBytes class takes a password, a salt, and an iteration count, and then generates keys through calls to the GetBytes method. Repeated calls to this method will not generate the same key; ...

The emphasis is mine, but is intended to point out that it is a key generator not just a hasher. So, in your "Fails with Database" code you have this block:

Dim key As Rfc2898DeriveBytes = New Rfc2898DeriveBytes(_sharedSecret, _salt)
...
Using myAes As Aes = Aes.Create()
    ...
    myAes.Key = key.GetBytes(myAes.KeySize / 8)
    myAes.IV = key.GetBytes(myAes.BlockSize / 8)
    ...
End Using

The Key and IV generated will not be the same. This is good, but probably not what you intended. The same block is used later to decrypt what was read back from the db. The Key and IV used to decrypt will also be different from each other, but more importantly they will not be the same Key and IV used to encrypt!

This is a fatal flaw, and causes the error. It seems like a misleading error message, but it is simple to reproduce/fix. You need a new PBKDF to 'start over' in the decryption part, or Reset the existing one:

' after this line...
Dim decrypted As String = ""
' ...add this:
key As Rfc2898DeriveBytes = New Rfc2898DeriveBytes(_sharedSecret, _salt)
' or
'key.Reset()

It is more likely that the real code would not be doing a roundtrip in the same method, and using a new Rfc2898DeriveByteswould be created.

Finally, note that the IV and Salt should be unique and random for effective encryption. There really isn't much point in encrypting each row with the same password, key, salt and IV beyond the illusion of security. Someone only has to crack the PW for one row to get the data for all the rows.

Because Rfc2898DeriveBytes is deterministic, it can play a role in generating the IV and Salt (for the data to be encrypted) without having to save them or use static ones.

Answer 2

Generally a padding error really means that the key, encrypted data or options are incorrect, not the padding, per se.

Make sure the key is exactly the correct length, if not the implementation will use unspecified bytes. Make the key exactly a correct key size: 128, 192 or 256 bits (16, 24 or 32 bytes).

Make sure the key, data and IV do not have an incorrect encoding, encryption is based on 8-bit bytes with no encoding. Some libraries will accept and produce other encodings such as Base64 and hexadecimal, check the documentation carefully.

Hex dump the key, IV, encrypted data and decrypted data and add samples of these to the question.

Encryption function calls are actually simple, if you supply the correct (and full) inputs the output will be correct. If the results check the inputs carefully.

UPDATE:
The IV and key are not the same for both encryption and decryption, they must be the same.

One way to ensure the IV is the same is to pre-pend it to the encrypted data on encryption and on decryption recover it from the encrypted data and skip it when decryption the encrypted data.

The key must be pre-shared between the encryption and decryption code in some manner prior to the decryption.