Output of crc32b in PHP is not equal to Python

Question

I'm trying to convert PHP snippet into Python3 code but outputs of print and echo are different.

You can see it in the step 1.

Do you know where is the problem? I'm attaching input arrays too but I think they are equal.

�W2+ vs ee7523b2

EDIT

When I switch raw from TRUE to FALSE, outputs of 1st step are the same. $d = strrev(hash("crc32b", $d, FALSE)) . $d

But the problem is that I have to convert PHP to Python, not the opposite because, then I'm usit in the step 2 which I need to have equal output.

PHP OUTPUT (CMD)

0 ->    1   1   100 EUR 20190101    11111111                Faktúra 1   SK6807200002891987426353        0   0
1 -> �W2+   1   1   100 EUR 20190101    11111111                Faktúra 1   SK6807200002891987426353        0   0
2 -> 00004e00007715c242b04d5014490af1445dd61c1527ddc5f4461ca5886caf63fd8fbcf7df69c2035760ecb28d8171efdb409c0206996498ea7921e715172e60c210f923f070079ffba40000

PYTHON OUTPUT

-------
0 ->    1   1   100 EUR 20190101    11111111                Faktúra 1   SK6807200002891987426353        0   0
1 -> ee7523b2   1   1   100 EUR 20190101    11111111                Faktúra 1   SK6807200002891987426353        0   0
2 -> b'00006227515c7830302762275c783030325c7865305c7864386a34585c7862346d5c7838665c7865625c7863315c786266625c7839625c786339675c786332785c7831645c7862392c415c7862625c7831645c78663770365c786463735c786236572d606c225c7865355c7865635c7831345c7863655c786331205c7830635c7831315c7861375c7839345c7864665c7865635c7830365c7831652c22265c7866355c7862335c7866345c78616145585c7861625c7866395c7839615c7839645c7865645c7864625c7830305c7864355c7861643b5c7865365f5c7866645c786533405c78303027'

PHP

<?php
$suma = "100";
$datum = "20190101";
$varsym = "11111111";
$konsym = "";
$specsym = "";
$poznamka = "Faktúra";
$iban = "SK6807200002891987426353";
$swift = "";

$d = implode("\t", array(
    0 => '',
    1 => '1',
    2 => implode("\t", array(
        true,
        $suma,                      // SUMA
        'EUR',                      // JEDNOTKA
        $datum,                 // DATUM
        $varsym,                    // VARIABILNY SYMBOL
        $konsym,                        // KONSTANTNY SYMBOL
        $specsym,                       // SPECIFICKY SYMBOL
        '',
        $poznamka,                  // POZNAMKA
        '1',
        $iban,  // IBAN
        $swift,                 // SWIFT
        '0',
        '0'
    ))
));
// 0
echo "0 -> ".$d."\n";
$d = strrev(hash("crc32b", $d, TRUE)) . $d;
// 1
echo "1 -> ".$d."\n";
$x = proc_open("/usr/bin/xz '--format=raw' '--lzma1=lc=3,lp=0,pb=2,dict=128KiB' '-c' '-'", [0 => ["pipe", "r"], 1 => ["pipe", "w"]], $p);
fwrite($p[0], $d);
fclose($p[0]);
$o = stream_get_contents($p[1]);
fclose($p[1]);
proc_close($x);

$d = bin2hex("\x00\x00" . pack("v", strlen($d)) . $o);
// 2
echo "2 -> ".$d."\n";
?>

PYTHON

    def crc32b(x):
        h = zlib.crc32(x)
        x='%08X' % (h & 0xffffffff,)
        return x.lower()

    t = "\t"
    gen = t.join(["1",
                  "100", # SAME VARIABLES 
                  "EUR",
                  "20190101",
                  "11111111",
                  "",
                  "",
                  "",
                  "Faktúra",
                  "1",
                  "SK6807200002891987426353",
                  "",
                  "0",
                  "0"]
                 )

    d = t.join([
        "", "1", gen])
    # 0
    print(f"0 -> {d}")
    hashD = crc32b(d.encode()) # OK

    hashD = hashD[::-1]
    # hashD = str(binascii.unhexlify(hashD))
    d = hashD + d
    # 1
    print(f"1 -> {d}")
    args = shlex.split("xz '--format=raw' '--lzma1=lc=3,lp=0,pb=2,dict=128KiB' -c -")
    process = subprocess.Popen(args, shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                               stderr=subprocess.PIPE)
    output = process.communicate(d.encode())

    pack = "\x00\x00" + str(struct.pack("H", len(d))) + str(output[0])

    d = binascii.hexlify(pack.encode())
    # 2
    print(f"2 -> {d}")

Answer 1

Didn't work with PHP.
According to [Python.Docs]: zlib.crc32(data[, value]) (emphasis is mine):

Computes a CRC (Cyclic Redundancy Check) checksum of data. The result is an unsigned 32-bit integer.

You are making a confusion between:

• Its value - which can also be seen as an ASCII string of length 4

• The textual representation of its value (in base 16) - which is a string of length 8

>>> crc = 0x2B3257EE  # The value returned by zlib.crc32 for your text
>>> type(crc), crc
(<class 'int'>, 724719598)
>>>
>>> [chr((crc >> shift_bits) & 0xFF) for shift_bits in [0, 8, 16, 24]]
['î', 'W', '2', '+']

Notes:

• One way to do it is converting each of the number's 4 bytes into a char

• To get a byte from the uint32 value, the uint32 must be shifted to the right ([Python.Wiki]: BitwiseOperators) by the order of that byte in the value ([3, 2, 1, 0]) multiplied by 8 (number of bits in a byte)

  • Also, to get rid of the unwanted bytes (any of them except the right-most one), the resulting value is also anded with 0xFF (255)

• Due to little endianness, the bytes are converted to chars in reversed order (from right to left)

• The 1^st char ('î') looks different, but it's just a matter of representation (in my console vs. yours)

Integrating it into your code, you need to modify your crc32b function (and also remove any further processing on hashD) to:

def crc32b(x):
    crc = zlib.crc32(x)
    return "".join([chr((crc >> shift_bits) & 0xFF) for shift_bits in [0, 8, 16, 24]])

For more details on this general topic, check [SO]: Python struct.pack() behavior (@CristiFati's answer).

---

Update #0

Adding the version that starts from the hex representation:

>>> crc = 0x2B3257EE
>>> crc_hex = "{:08X}".format(crc)
>>> crc_hex
'2B3257EE'
>>>
>>> list(reversed([chr(int(crc_hex[2 * i] + crc_hex[2 * i + 1], 16)) for i in range(len(crc_hex) // 2)]))
['î', 'W', '2', '+']

• From my PoV, this is uglier, also it's inefficient (many back and forth conversions), but posting anyway as some folks have trouble working on bit operations

• The key point is to handle 2 hex chars at a time, and only reverse after conversion

Answer 2

you just have to remove the 3rd argument of the function hash()

if you set this argument to true, hash will return raw binary data and php is trying to parse it as a text string whereas you are expecting a hexadecimal result