Generated sha1 key in PBKDF2 Python do not match in .NET Rfc2898

Question

I have a python code that generates a PBKDF2 sha1 hash of a password using the hashlib.pbkdf2_hmac method. Then I use that password digest in a dotnet framework 4.5 program to verify it against the same password. The C# program returns false which suggests that the hash produced from the python program is incorrect.

The key is in this format: #iterations|salt|key. Then, I take that key and I try to verify it using a dotnet framework app using via method:

public static bool IsValid(string testPassword, string originalDelimitedHash)
{
    //extract original values from delimited hash text
    var originalHashParts = originalDelimitedHash.Split('|');
    var origIterations = Int32.Parse(originalHashParts[0]);
    var origSalt = Convert.FromBase64String(originalHashParts[1]);
    var originalHash = originalHashParts[2];

    //generate hash from test password and original salt and iterations
    var pbkdf2 = new Rfc2898DeriveBytes(testPassword, origSalt, origIterations, HashAlgorithmName.SHA1);
    byte[] testHash = pbkdf2.GetBytes(20);

    var hashStr = Convert.ToBase64String(testHash);
    if (hashStr == originalHash)
        return true;

    return false;
}

my python program:

from hashlib import pbkdf2_hmac
from base64 import b64encode
from os import urandom

def generate_password_hash(password:string):
    encodedPass = password.encode('utf8')

    random_bytes = urandom(20)

    salt = b64encode(random_bytes)
    iterations = 5000
    key = pbkdf2_hmac('sha1', encodedPass, salt, iterations, dklen=20)

    result = f'{iterations}|{salt.decode("utf-8")}|{binascii.hexlify(key).decode("utf-8")}'

    return result

So if my password is hDHzJnMg0O the resulting digest from the above python method would be something like 5000|J5avBy0q5p9R/6cgxUpu6+6sW7o=|2445594504c9ffb54d1f11bbd0b385e3e37a5aca

So if I take that and supply it to my C# IsValid method (see below) it returns false which means the passwords do not match

static void Main(string[] args)
{
    var pass = "hDHzJnMg0O";

    var hash = "5000|J5avBy0q5p9R/6cgxUpu6+6sW7o=|2445594504c9ffb54d1f11bbd0b385e3e37a5aca";
    var isValid = IsValid(pass, hash); // returns False
}

Answer 1

The Python code:

• uses b64encode(random_bytes) as salt for the PBKDF2 call. This is rather unusual (but not a bug). Typically the raw data, i.e. random_bytes, is applied as salt and passed to the PBKDF2 call. With the Base64 encoding only the string would be created.
• hex encodes the key (i.e. the return value of the PBKDF2 call).

The C# code is different in these points and:

• uses the raw data (i.e. random_bytes from the Python side) for the PBKDF2 call, i.e. the salt from the Python side is Base64 decoded.
• Base64 encodes the key (i.e. the return value of the PBKDF2 call)

Changes in the C# code for compatibility with the Python code (of course the changes could also be made in the Python code, but the Python code seems to be the reference):

...
var origSalt = Encoding.UTF8.GetBytes(originalHashParts[1]); // Convert.FromBase64String(originalHashParts[1]);
...
var hashStr = Convert.ToHexString(testHash); // Convert.ToBase64String(testHash);
...

For the latter, Convert.ToHexString() was used, which is available since .NET 5. For other .NET versions see e.g. here.

Furthermore, since the hex encoded values are compared and the different implementations are not standardized regarding lower (e.g. binascii.hexlify(key)) and upper case letters (e.g. Convert.ToHexString(testHash)), it is more robust to convert both strings uniformly, e.g.:

if (hashStr.ToUpper() == originalHash.ToUpper())
    return true;

With these changes, validation with the C# code works.

---

Edit (with regard to the change in the Python code addressed in the comment):

If in the Python code random_bytes is used as salt and the salt is Base64 encoded for concatenation, then in the C# code the Base64 encoded salt must be Base64 decoded again (as in the original C# code).