Python: how to get a 4-byte sized integer from a 4-byte byte array?

Question

Here is a simple Python (version 3.4) code I've written to get a 32bit sized integer (int type I would assume) from an array of 4 bytes:

import binascii
import socket   
import struct
import array
import pickle
import ctypes
import numpy
import sys

float_val = 1.0 + 0.005
print(float_val)

packed = struct.pack('f', float_val)
print(len(packed)) 

tempint2 = struct.unpack(">I", packed)[0]
tempint3 = struct.unpack_from(">I", packed)[0]
tempint4 = int.from_bytes(packed, byteorder='big', signed=False)

print(sys.getsizeof(tempint2))
print(tempint2)
print(sys.getsizeof(tempint3))
print(tempint3)
print(sys.getsizeof(tempint4))
print(tempint4)

However, none of the attempts (tempint2/tempint3/tempint4) gives the value I expected (4-byte size integer). Somehow, the size is all 18 bytes (sys.getsizeof() function result). Can you tell me how to get the expected answer (4-byte or 32bit size integer)?

`sys.getsizeof() ` includes the overhead of the python object ... you can use numpy to create an array of actual 4 byte integers ... or just deal with the fact that is is 4 bytes of number data and 12 bytes of python object overhead (this is almost definately fine....) is it not giving you the actual value you expect for the number? if not what do you expect vs what do you get? — Joran Beasley, Nov 05 '15 at 22:50
This code sniffet is from another python program where I created 4-byte int array by using numpy. What I want to do is like this (in C): float f = 1.005; unsigned int a = *(unsigned int *)(&f); However, the python complains that the numpy 32bit int cannot accomodate the int came from the code sniffet above. — InKwon Park, Nov 05 '15 at 22:54
capital I is unsigned 4 byte int ... if you want negative values you will need lowercase i (I think) — Joran Beasley, Nov 05 '15 at 22:56
If you're trying to get a 32-bit float, why aren't you asking `struct` to give you one of those instead? — Mark Ransom, Nov 05 '15 at 23:43
Possible duplicate of [convert a string of bytes into an int (python)](http://stackoverflow.com/questions/444591/convert-a-string-of-bytes-into-an-int-python) — jpmc26, Nov 06 '15 at 02:23
Perhaps, my question was not too clear. Let me rephrase it: how to put python native int variable into numpy intc type variable? — InKwon Park, Nov 06 '15 at 02:23

3442 · Answer 1 · 2016-03-14T03:34:18.330

First of all, due to Python's... ahem... "magic", sys.getsizeof() won't return the length of a list, but the sizeof the whole datastructure as represented internally by the Python interpreter.

Now, the answer (for integers) is simply... (for all combinations of Python 2.x/Python 3.x and 32-bit/64-bit):

from math import ceil, floor, log

def minimumAmountOfBytesToHoldTheStuff(x):
    # Avoid math domain errors
    if x < 0:
        x = ~x

    # Avoid more math domain erros
    if x == 0:
        x = 1

    return int(ceil((floor(log(x, 2)) + 1 ) / 8))

def powersOfTwo():
    x = 1
    while True:
        yield x
        x *= 2

def minimumAmountOfBytesToHoldTheStuffOnRealMachines(x):
    bytes = minimumAmountOfBytesToHoldTheStuff(x)
    for power in powersOfTwo():
        if bytes <= power:
            return power

print(minimumAmountOfBytesToHoldTheStuffOnRealMachines(tempint))

Note: It appears that log(x, 2) breaks for x >= pow(2, 48) - 1, and so does the whole algorithm. This is probably an issue from the C library/the stupid floating-point accurracy errors, because log(n, x) in Python is translated into log(n) / log(x) in C.

Edit: This one is an optimized version for Python 3.x that is independent of bot floating-point and logarithmic operations, and thus is accurate on all situations...

from math import ceil

def minimumAmountOfBytesToHoldTheStuff(x):
    # Avoid math domain errors
    if x < 0:
        x = ~x

    # Avoid more math domain erros
    if x == 0:
        x = 1

    return int(ceil(x.bit_length() / 8))

The other functions are the same.

I hope this has led some light on you!

That's funny, `math.log(2**33, 2)` works fine for me in Python 2.7.5, returns `33.0`. — Mark Ransom, Nov 05 '15 at 23:41
@MarkRansom: Try out `math.log(2**48-1, 2)`. The function works out whenever `x` is a perfect power of 2, but not otherwise if `x >= pow(2, 48) - 1`. **Edit**: Sorry, I originally stated the barrier was at `2^32`, whereas it is at `2^48`. — 3442, Nov 06 '15 at 00:36
@KemyLand: Note that Python 3's [`math` module offers a `log2` function](https://docs.python.org/3/library/math.html#math.log2) that computes more accurately. And for computing bits required to hold a value, it provides the even more useful/efficient `bit_length()` method on `int`s. Since the OP is using Python 3.4, you could use `x.bit_length()` for speed and accuracy. — ShadowRanger, Nov 06 '15 at 02:18
@ShadowRanger: Thanks, I didn't noticed the OP was using 3.4, so I'll leave the code as-is for 2.x readers and add a modified/optimized version for 3.4. — 3442, Nov 06 '15 at 02:21

Python: how to get a 4-byte sized integer from a 4-byte byte array?

1 Answers1