2

I have created multiple filters for a Dataframe:

filt1 = ~df["message"].str.contains("<Media omitted>", na=False),
filt2 = ~df["message"].str.contains("http://", na=False),
filt3 = ~df["message"].str.contains("Dropped pin", na=False), 

I can filter the dataframe using:

df[filt1 & filt2 & filt3]

But as I add more filters this seems like a stupid way to filter. How do I apply multiple filters to a dataframe?

I tried adding each filter to a list doing df[filterlist] and df[*filterlist] but these do not work.

gazm2k5
  • 449
  • 5
  • 14

2 Answers2

3

You can use np.logical_and.reduce:

filterlist = [filt1, filt2, filt3]

df[np.logical_and.reduce(filterlist)]

Or concat with DataFrame.all for test all Trues per rows:

df[pd.concat(filterlist, axis=1).all(axis=1)]

If possible use | for regex or:

filt = ~df["message"].str.contains("<Media omitted>|http://|Dropped pin", na=False)
jezrael
  • 822,522
  • 95
  • 1,334
  • 1,252
0

You can use join to merge list for regex

df[~df.message.str.contains('|'.join(filterlist))]
Dishin H Goyani
  • 7,195
  • 3
  • 26
  • 37