Convert Bitstring (String of 1 and 0s) to numpy array

Question

I have a pandas Dataframe containing 1 columns which contains a string of bits eg.'100100101'. i want to convert this string into an numpy array.

How can I do that?

EDIT:

Using

features = df.bit.apply(lambda x: np.array(list(map(int,list(x)))))
#...
model.fit(features, lables)

leads to an error on model.fit:

ValueError: setting an array element with a sequence.

The Solution that works for my case i came up with due to marked answer:

for bitString in input_table['Bitstring'].values:
    bits = np.array(map(int, list(bitString)))
    featureList.append(bits)
features = np.array(featureList)
#....
model.fit(features, lables)

jedwards · Accepted Answer · 2015-03-18T21:05:45.707

For a string s = "100100101", you can convert it to a numpy array at least two different ways.

The first by using numpy's fromstring method. It is a bit awkward, because you have to specify the datatype and subtract out the "base" value of the elements.

import numpy as np

s = "100100101"
a = np.fromstring(s,'u1') - ord('0')

print a  # [1 0 0 1 0 0 1 0 1]

Where 'u1' is the datatype and ord('0') is used to subtract the "base" value from each element.

The second way is by converting each string element to an integer (since strings are iterable), then passing that list into np.array:

import numpy as np

s = "100100101"
b = np.array(map(int, s))

print b  # [1 0 0 1 0 0 1 0 1]

Then

# To see its a numpy array:
print type(a)  # <type 'numpy.ndarray'>
print a[0]     # 1
print a[1]     # 0
# ...

Note the second approach scales significantly worse than the first as the length of the input string s increases. For small strings, it's close, but consider the timeit results for strings of 90 characters (I just used s * 10):

fromstring: 49.283392424 s
map/array:   2.154540959 s

(This is using the default timeit.repeat arguments, the minimum of 3 runs, each run computing the time to run 1M string->array conversions)

Note that `np.array(map(int, s))` is sufficient - it isn't required to build a `list` first... Also, it's not exactly intuitive, but `np.fromstring(s, 'i1') - 48` is about 50% faster... — Jon Clements, Mar 18 '15 at 06:01
@JonClements performance turns out to be much worse for longer lists -- thanks for the note! — jedwards, Mar 18 '15 at 21:06
@JonClements I don't think this is still true in Python 3.x. Now that map returns a map object (iterator), you either have to wrap in `list` or use `np.fromiter(map(int, s))` — Benjamin Doughty, Apr 14 '18 at 17:37

score 2 · Answer 2 · answered Mar 17 '15 at 09:18

2

One pandas method would be to call apply on the df column to perform the conversion:

In [84]:

df = pd.DataFrame({'bit':['100100101']})
t = df.bit.apply(lambda x: np.array(list(map(int,list(x)))))
t[0]
Out[84]:
array([1, 0, 0, 1, 0, 0, 1, 0, 1])

answered Mar 17 '15 at 09:18

EdChum

376,765
198
813
562

score 0 · Answer 3 · answered Feb 19 '19 at 12:19

Check unpackbits

>>> np.unpackbits(np.array([int('010101',2)], dtype=np.uint8))
array([0, 0, 0, 1, 0, 1, 0, 1], dtype=uint8)

And more generally:

>>> a = np.array([[2], [7], [23]], dtype=np.uint8)
>>> a
array([[ 2],
       [ 7],
       [23]], dtype=uint8)
>>> b = np.unpackbits(a, axis=1)
>>> b
array([[0, 0, 0, 0, 0, 0, 1, 0],
       [0, 0, 0, 0, 0, 1, 1, 1],
       [0, 0, 0, 1, 0, 1, 1, 1]], dtype=uint8)

If you need more than 8 bits, check out How to extract the bits of larger numeric Numpy data types

Convert Bitstring (String of 1 and 0s) to numpy array

3 Answers3

Linked