I have two date columns(format eg: 5/12/21) and need to find difference between them in days using pyspark. I tried but getting error.Need with Withcolumn.
date1 - date2 = result(in integer format)
Asked
Active
Viewed 326 times
0
-
???any leads??? – datatatata Aug 23 '21 at 20:31
-
1What have you been doing so far ? Could you provide some piece of code ? – Val Berthe Aug 23 '21 at 20:32
-
1I believe this is a duplicate: https://stackoverflow.com/questions/44020818/how-to-calculate-date-difference-in-pyspark – WIT Aug 23 '21 at 20:44
1 Answers
0
If you have column "date1" and column "date2"
import pyspark.sql.functions as F
df = df.withColumn("numDaysDiff", F.datediff(F.col("date1"), F.col("date2")))
WIT
- 1,043
- 2
- 15
- 32
-
-
If they are, then no need. If not, then you can do `F.col("date1").cast("date")` – WIT Aug 23 '21 at 20:38
-
I believe this is a duplicate, so before I edit, maybe review this link: https://stackoverflow.com/questions/44020818/how-to-calculate-date-difference-in-pyspark – WIT Aug 23 '21 at 20:45
-