Passing date as parameter in Pyspark (Date is being converted into Integer when passed as argument)

Asked Jun 28 '18 at 12:16

Active Jun 28 '18 at 12:27

Viewed 752 times

I am passing date as parameter in a script.

"process_date":"2018-06-10"

And then trying to print this

from datetime import datetime, timedelta
import sys
process_date=datetime.strptime(sys.argv[1], '%Y-%m-%d')
print(process_date)

But I am getting an error

time data '41685' does not match format '%Y-%m-%d'

The date parameter is getting converted to integer. How can I solve this ?

edited Jun 28 '18 at 12:27

asked Jun 28 '18 at 12:16

Bitanshu Das

Possible duplicate of [Convert pyspark string to date format](https://stackoverflow.com/questions/38080748/convert-pyspark-string-to-date-format) – Ramesh-X Jun 28 '18 at 12:18
No.Its different – Bitanshu Das Jun 28 '18 at 12:26
how the python file is called in the script..? – Ramesh-X Jun 28 '18 at 12:30
I am trying to pass it to a notebook. – Bitanshu Das Jun 28 '18 at 12:32
creating a databricks job which reads it as date – Bitanshu Das Jun 28 '18 at 12:32
@BitanshuDas what is the expected output? – User12345 Jun 28 '18 at 13:57
Can you add the output of `print(sys.argv[1])` before the call to `strptime`? How do you know the correct value is being passed in? – pault Jun 28 '18 at 14:49
print(sys.argv[1]) gives me 39337 – Bitanshu Das Jun 29 '18 at 06:22

0 Answers0