8

Question

How can I perform bitwise operations in Pandas?

How & works on integers

On integers the & operator performs a bitwise mask

>>> mask = 0b1100  # 4 and 8 bits on
>>> 7 & mask
4

How & works in Pandas

Is there some way to perform bitwise masking operations in Pandas? The & operator does something else.

>>> df = DataFrame([1, 2, 3, 4, 5, 6, 7, 8], columns=['data'])
>>> df.data & mask
0    False
1    False
2    False
3     True
4     True
5     True
6     True
7     True
Name: data, dtype: bool
MRocklin
  • 55,641
  • 23
  • 163
  • 235

1 Answers1

11
In [184]: df = pd.DataFrame([1, 2, 3, 4, 5, 6, 7, 8], columns=['data'])

In [185]: mask = 0b1100

In [186]: np.bitwise_and(df['data'], mask)
Out[186]: 
0    0
1    0
2    0
3    4
4    4
5    4
6    4
7    8
Name: data, dtype: int64

It even returns a Series -- pretty cool!

unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • 6
    Alternatively, `df["data"].values & 0b1100`. – DSM Mar 06 '14 at 17:26
  • 2
    One more suggestion `df['data'] = df['data'].apply(lambda x: x & mask)` – Shravan Apr 18 '14 at 06:42
  • 4
    If performance is a concern, I would recommend not exploring the usage of apply & lambda. On a Series of 238517 records, using the timeit module, I measured ~500 microseconds to both right shift & mask values using the accepted or .values & mask suggestion. However, the apply+lambda approach consistently would take around 1 second. Jumpin from microsecond measurements to seconds. – John Apr 05 '17 at 14:34