I have a dataset with a column that represents date time the following format: yyyy-MM-dd HH:mm:ss.SSS
I want to convert this date-time in a Unix timestamp at millisecond granularity. I tried the following:
filteredDS.withColumn(TIMESTAMP, unix_timestamp(filteredDS(TIMESTAMP)))
but this just gives me the timestamp in seconds and not milliseconds. How do I get the timestamp in millis from a date column?