Convert String to timestamp in Sparklyr

Question

I'd like to convert String to time stamp, I used the below code but it added 4 hours to the timestamp, I really appreciate if you could help me to figure it out the issue

I used Sparklyr.

fds<- fds %>%  
  mutate(time2 = from_unixtime(
    unix_timestamp(transaction_time, "yyyy-MM-ddHH:mm:ss"), "yyyy-MM-dd HH:mm:ss"))
fds<- fds %>% mutate(transaction_time_2 = to_timestamp(time2))

transaction_time is string with this format '2018-05-3016:00:40' by using from_unixtime, it has been converted to ''2018-05-30 16:00:40' and by using to_timestamp it has been converted to 2018-05-30 20:00:40. everything is correct except 20 hour, I still expected 16.

score 2 · Accepted Answer · answered Jun 07 '19 at 16:53

2

I encountered the same problem, I think it relates to the timezone settings try the following at the top of your program, then the problem should disappear:

sparklyr::invoke_static(sc, "java.util.TimeZone",  "getTimeZone", "GMT") %>%
  sparklyr::invoke_static(sc, "java.util.TimeZone", "setDefault", .)

check out this answer for more details :

changing the JVM timezone in sparklyr

answered Jun 07 '19 at 16:53

Mouad_Seridi

2,666
15
27

1

thanks for your help, it worked @Mouad_S – Mrygh Jun 07 '19 at 17:01
1

This was a lifesaver, thank you! None of the other solutions in similar questions seemed to work like this did. – J. Blauvelt Feb 19 '20 at 03:48

Convert String to timestamp in Sparklyr

1 Answers1