I have a column of dates in a pyspark dataframe in the format 01-01-1999 (dd-mm-yyyy)and I would like to change the date format of the entire column to 1999-01-01 (yyyy-mm-dd). Any help?
Asked
Active
Viewed 384 times
-1
-
`datetime` and `date` in Python have no format, they're binary values. Either you have a string instead of date, or you confuse how the `date` is displayed on some UI with the actual way a `date` is stored. – Panagiotis Kanavos Sep 29 '22 at 13:55
-
see [`to_date`](https://spark.apache.org/docs/3.3.0/api/python/reference/pyspark.sql/api/pyspark.sql.functions.to_date.html#pyspark-sql-functions-to-date) function in pyspark – samkart Sep 29 '22 at 14:34
1 Answers
0
In pyspark we have this function:
date_format(dateExpr,format)
It converts a date/timestamp/string to a value of string in the format specified by the date format given by the second argument. You can try ths one:
df.select(date_format('your_column_date', "yyyy-MM-dd")).show()
ALI SALMI
- 1
- 1
-
-
-
[`date_format`](https://spark.apache.org/docs/3.3.0/api/python/reference/pyspark.sql/api/pyspark.sql.functions.date_format.html#pyspark-sql-functions-date-format) can only format if it is already in default format ('yyyy-MM-dd'). – samkart Sep 29 '22 at 15:16