0

pyspark 2.3.1

my rows to col1 should only contain integers. I am trying to filter out any row that have even one character. How can I do this in pyspark?

I've tried

df.select('col1').filter(df.col1.rlike(^[a-zA-Z])) 

however rows that contain alphabet also contain integers therefore not filtered.

How can I do this?

haneulkim
  • 4,406
  • 9
  • 38
  • 80

1 Answers1

1

You can try to select pure digital rows.

df = df.filter('col1 rlike "^[0-9]+$"')
df.show(truncate=False)
过过招
  • 3,722
  • 2
  • 4
  • 11