How to pack & unpack 64 bits of data?

Question

I have a 64-bit data structure as follows:

HHHHHHHHHHHHHHHHGGGGGGGGGGGGFFFEEEEDDDDCCCCCCCCCCCCBAAAAAAAAAAAA

A: 12 bits (unsigned)
B: 1 bit
C: 12 bits (unsigned)
D: 4 bits (unsigned)
E: 4 bits (unsigned)
F: 3 bits (unsigned)
G: 12 bits (unsigned)
H: 16 bits (unsigned)

Using Python, I am trying to determine which module (preferably native Python 3.x) I should be using. I am looking at BitVector but having trouble figuring some things out.

For ease of use, I want to be able to do something like the following:

# To set the `A` bits, use a mapped mask 'objectId'
bv = BitVector(size=64)
bv['objectId'] = 1

I'm not sure that BitVector actually works the way I want it to. Whatever module I end up implementing, the data structure will be encapsulated in a class that reads and writes to the structure via property getters/setters.

I will also be using constants (or enums) for some of the bit values and it would be convenient to be able to set a mapped mask using something like:

# To set the 'B' bit, use the constant flag to set the mapped mask 'visibility'
bv['visibility'] = PUBLIC  
print(bv['visibility']) # output: 1  

# To set the 'G' bits, us a mapped mask 'imageId'
bv['imageId'] = 238

Is there a Python module in 3.x that will help me achieve this goal? If BitVector will (or should) work, some helpful hints (e.g. examples) would be appreciated. It seems that BitVector wants to force everything to an 8-bit format which is not ideal for my application (IMHO).

What format do you receive the 64-bits of data in? String, byte string, integer value, or ? — martineau, Feb 06 '16 at 18:26
The data is retrieved and sent as shown in the format above - essentially as a 64-bit numeric value that I then operate via the pseudo-API in the question. — IAbstract, Feb 06 '16 at 18:29
I've written stuff that extracts bit fields from larger structs. I don't recall any special modules to help. I recall a lot of use of ``>>`` and a unit test library with 100% coverage — Vorsprung, Feb 06 '16 at 18:42
Take a look at question [_Does Python have a bitfield type?_](http://stackoverflow.com/questions/142812/does-python-have-a-bitfield-type). — martineau, Feb 06 '16 at 18:43
Try [bitarray](https://pypi.python.org/pypi/bitarray/) instead of BitVector, it's pure C. — L3viathan, Feb 06 '16 at 18:52
@L3viathan: if you provide an answer using `bitarray` with an example modifying the value of a long then I'll accept that as an answer. Plus I'm trying to figure out how to install it on my Windows machine. — IAbstract, Feb 08 '16 at 17:13

score 0 · Accepted Answer · answered Feb 08 '16 at 21:20

Based on the recommendation to use bitarray I have come up with the following implementation with two utility methods:

def test_bitvector_set_block_id_slice(self):
    bv = bitvector(VECTOR_SIZE)
    bv.setall(False)

    print("BitVector[{len}]: {bv}".format(len=bv.length(),
                                          bv=bv.to01()))
    print("set block id: current {bid} --> {nbid}".format(bid=bv[52:VECTOR_SIZE].to01(),
                                                          nbid=inttobitvector(12, 1).to01()))

    # set blockVector.blockId (last 12 bits)
    bv[52:VECTOR_SIZE] = inttobitvector(12, 1)

    block_id = bv[52:VECTOR_SIZE]

    self.assertTrue(bitvectortoint(block_id) == 1)
    print("BitVector[{len}] set block id: {bin} [{val}]".format(len=bv.length(),
                                                                bin=block_id.to01(),
                                                                val=bitvectortoint(block_id)))
    print("BitVector[{len}]: {bv}".format(len=bv.length(),
                                          bv=bv.to01()))
    print()

# utility methods
def bitvectortoint(bitvec):
    out = 0
    for bit in bitvec:
        out = (out << 1) | bit

    return out


def inttobitvector(size, n):
    bits = "{bits}".format(bits="{0:0{size}b}".format(n,
                                                      size=size))

    print("int[{n}] --> binary: {bits}".format(n=n,
                                               bits=bits))

    return bitvector(bits)

The output is as follows:

BitVector[64]: 0000000000000000000000000000000000000000000000000000000000000000
int[1] --> binary: 000000000001
set block id: current 000000000000 --> 000000000001
int[1] --> binary: 000000000001
BitVector[64] set block id: 000000000001 [1]
BitVector[64]: 0000000000000000000000000000000000000000000000000000000000000001

If there are improvements to the utility methods, I am more than willing to take some advice.

How to pack & unpack 64 bits of data?

1 Answers1