3

I have a bytes element. My word size is 1, a single byte. The contents can be b'\xff\xff\x01' meaning [-1, -1, 1].

I want to convert it to the int representation from the bytes form. Logically my attempt is:

ints = [int.from_bytes(j, byteorder='little', signed=True) for j in b'\xff\xff\x01']

TypeError: cannot convert 'int' object to bytes

However this does not work as the for j in bytes() converts a bytes element directly into an int j. This is however an unsigned conversion, I need signed. How do I convert my bytes, byte by byte, into a signed integer.

Tarick Welling
  • 3,119
  • 3
  • 19
  • 44
  • "This is however an unsigned conversion, I need signed." Did you try doing the math to convert to 2s complement? – Karl Knechtel Sep 20 '22 at 14:56
  • `bytes` *are* an array of ints, just presenting as sort-of ASCII string when printed as a whole. – deceze Sep 20 '22 at 14:56
  • 1
    @quamrana Does that *really* answer the question? – deceze Sep 20 '22 at 15:02
  • 2
    `ints = [int.from_bytes(bytes([j]), byteorder='little', signed=True) for j in b'\xff\xff\x01']` – realSamy Sep 20 '22 at 15:08
  • @realSamy thanks, that also works. Really useless that such a hacky solution is needed to keep the correct type info – Tarick Welling Sep 20 '22 at 15:10
  • @KarlKnechtel this is python, why would I need to do bit twiddling when the `from_bytes` can do it. – Tarick Welling Sep 20 '22 at 15:12
  • 1
    @realSamy feel free to post your answer, I'll accept it as it is a better one than the hack I came up with – Tarick Welling Sep 20 '22 at 15:12
  • @deceze: Seems I was wrong. If only realSamy would post his answer. – quamrana Sep 20 '22 at 15:15
  • 1
    "Really useless that such a hacky solution is needed to keep the correct type info" - Python 3.x correctly recognizes (aside from some very silly looking legacy methods like `zfill`) that bytes are **not text**. Python doesn't have a separate "character" type, so it makes sense for indexing into a string to give another string. Bytes are raw data; an individual byte does have a natural representation as the corresponding numeric value - which in Python is spelled `int`. Intuitively, indexing into a sequence that is "one-dimensional" ought to give a scalar value. 3.x fixes this. – Karl Knechtel Sep 20 '22 at 15:17

4 Answers4

3

However this does not work as the for j in bytes() converts a bytes element directly into an int j.

As you've noticed, the bytes is already an iterable over integer values. If we have a lot of data, it would be more memory efficient to keep the bytes as is, and convert values on demand. We can simply do the math to convert the unsigned interpretation of a byte to the corresponding signed representation:

def signed_byte(b):
    return b - 256 if b >= 128 else b

And wrap the indexing process:

def get_signed(data, index):
    return signed_byte(data[index])

If we want or need to do all the conversion ahead of time, that feeds directly into the list comprehension:

ints = [signed_byte(b) for b in b'\xff\xff\x01']

On the other hand, we can reframe the question: to get a bytes object from the original data as a subsequence (like how it worked in 2.x), we can use a 1-element slice instead; or we can wrap the int value into a new bytes. The former will be tricky to adapt to the original code, but the latter is trivial:

ints = [int.from_bytes(bytes([b]), byteorder='little', signed=True) for b in b'\xff\xff\x01']
Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
1

Another way, using Numpy:

>>> import numpy as np
>>> np.frombuffer(b'\xff\xff\x01', dtype=np.int8)
array([-1, -1,  1], dtype=int8)
Karl Knechtel
  • 62,466
  • 11
  • 102
  • 153
0

This solution is idiotic but at least it works.

data = b'\xff\xff\x01'
result = list(struct.unpack('b' * len(data), data))
Tarick Welling
  • 3,119
  • 3
  • 19
  • 44
0

I don't know if it still helps! I saw your comment just now :)

ints = [int.from_bytes(bytes([j]), byteorder='little', signed=True) for j in b'\xff\xff\x01']
realSamy
  • 189
  • 6