Byte conversion from hash string, java vs python

Question

I have issue with converting a pure java Curve25519 function to Python equivalent, the specific problem is related to digest function that convert hash string to byte equivalent, the java implementation :

Data example :

sP = "this is a sample of secret pass phrase for test purpose"

/**
 * Calculate the SHA-256 hash of a string
 *
 * @param       input           Data to be hashed
 * @return                      The hash digest
 */
public static byte[] singleDigest(String input) {
    byte[] bytes;
    try {
        bytes = singleDigest(input.getBytes("UTF-8"));
        System.out.println("bytes"+bytes);
        System.out.println("before singleDigest"+input);
    } catch (UnsupportedEncodingException exc) {
        bytes = null;
    }
    return bytes;
}

/**
 * Calculate the SHA-256 hash of a byte array
 *
 * @param       input           Data to be hashed
 * @return                      The hash digest
 */
public static byte[] singleDigest(byte[] input) {
    byte[] bytes;
    synchronized (digest) {
        digest.reset();
        bytes = digest.digest(input);

        StringBuffer sb = new StringBuffer();
        for (int i = 0; i < bytes.length; i++) {
            System.out.println(bytes[i]);
            sb.append(String.format("%02x", bytes[i] & 0xFF));
        }
        System.out.println(sb.toString());
    }
    return bytes;
}

Generate this byte output :

byteOutput :82, -57, 124, 58, -105, 76, 123, 3, 119, -21, 121, 71, -54, 73, -75, 54, 31, -33, -49, -68, -16, -19, 125, -61, -116, -82, 96, 50, -35, -119, -28, 25

with positive and negative numbers, on python side, I use this function to convert a hashed string to byte :

secret = sha256(sP.encode('utf-8')).hexdigest()
sct = ParseHexString(secret).getData()

class ParseHexString(object):

    def __init__(self, hex):
        """

        """
        if not isinstance(hex, str):
            return None
        else:

            self.hex = hex
            self.listOfByte = None

        self.run()

    def run(self):

        if len(self.hex)&0x01 == 1:
            return None

        lbytes = []
        for i in range(0, len(self.hex), 2):
            op1, op2 = self.hex[i:i + 2]
            oop1 = ord(op1)
            oop2 = ord(op2)

            char1 = oop1 - 0x57 if oop1 > 0x60 else oop1 - 0x30
            char2 = oop2 - 0x57 if oop2 > 0x60 else oop2 - 0x30

            if (char1 < 0 or char2 < 0 or char1 > 15 or char2 > 15):
                print("Invalid hex number ",op1,op2)

            lbytes.append(((char1 << 4) + char2))

        self.listOfByte = lbytes

    def getData(self):
        return self.listOfByte

I used lists here instead of mutable bytearray because of internal representation of bytearray (0 - 256), and the output for some hashed string is different and is only positive integers :

82, 199, 124, 58, 151, 76, 123, 3, 119, 235, 121, 71, 202, 73, 181, 54, 31, 217, 207, 188, 240, 237, 125, 196, 140, 174, 96, 50, 221, 137, 228, 25

I noticed that (256 - JavaByteOutput[x] == PythonByteOtput[x]) when JavaByteOutput is a negative number, the question is, how I can modify my ParseHexString class to have equivalent positive/negative output, and I want pure python code and no libraries.

Thanks for any kind answer. Regards Alex

score 1 · Answer 1 · answered Dec 06 '17 at 14:52

1

The whole point here: Java only knows signed types.

This means: although the Java byte is also about 8 bits, the range is from -128 to 127. Unsigned bytes go from 0 to 255.

That is all there is to this. The only thing that really matters is that you are consistent about how the values dealt with. See here for some further thoughts on this topic.

answered Dec 06 '17 at 14:52

GhostCat

137,827
25
176
248

Thanks for explanation and suggestions! – Alex Dec 06 '17 at 15:08

unutbu · Accepted Answer · 2017-12-06T16:02:15.710

In Python, a % b always returns a value with the same sign as b between 0 (inclusive) and b (exclusive). In this case b is 256.

So to shift the values to the range [-128, 128), add 128 to each value, take the modulus base 256 then subtract 128:

x = [82, 199, 124, 58, 151, 76, 123, 3, 119, 235, 121, 71, 202, 73, 181, 54, 31, 217, 207, 188, 240, 237, 125, 196, 140, 174, 96, 50, 221, 137, 228, 25]

y = [(xi + 128) % 256 - 128 for xi in x]
# [82, -57, 124, 58, -105, 76, 123, 3, 119, -21, 121, 71, -54, 73, -75, 54, 31, -39, -49, -68, -16, -19, 125, -60, -116, -82, 96, 50, -35, -119, -28, 25]

Therefore, in ParseHexString.run define:

self.listOfByte = [(xi + 128) % 256 - 128 for xi in lbytes]

If NumPy were an option, you could instead simply view the unsigned 1-byte ints as signed 1-byte ints:

import numpy as np
x = np.array([82, 199, 124, 58, 151, 76, 123, 3, 119, 235, 121, 71, 202, 73, 181, 54, 31, 217, 207, 188, 240, 237, 125, 196, 140, 174, 96, 50, 221, 137, 228, 25], dtype='uint8')

x.view('i1')
# array([  82,  -57,  124,   58, -105,   76,  123,    3,  119,  -21,  121,
#          71,  -54,   73,  -75,   54,   31,  -39,  -49,  -68,  -16,  -19,
#         125,  -60, -116,  -82,   96,   50,  -35, -119,  -28,   25], dtype=int8)

Byte conversion from hash string, java vs python

2 Answers2