5

I have a dataframe with a TimeStamps column. I want to convert it to strings of local time, ie with daylight saving.

So I want to convert ts[0] below to "2015-03-30 03:55:05". Pandas seems to be aware of DST, but only when you call .values on the series.

Thanks

(Pdb) ts = df['TimeStamps']
(Pdb) ts
0   2015-03-30 02:55:05.993000
1   2015-03-30 03:10:20.937000
2   2015-03-30 10:09:19.947000
Name: TimeStamps, dtype: datetime64[ns]
(Pdb) ts[0]
Timestamp('2015-03-30 02:55:05.993000')
(Pdb) ts.values
array(['2015-03-30T03:55:05.993000000+0100',
   '2015-03-30T04:10:20.937000000+0100',
   '2015-03-30T11:09:19.947000000+0100'], dtype='datetime64[ns]')
jf328
  • 6,841
  • 10
  • 58
  • 82
  • what is your local timezone (e.g., `Europe/London`)? Is there a DST transition on 2015-03-30? – jfs Mar 31 '15 at 08:49
  • Hi, yes, DST started on 2015-03-29. The timestamp in UTC is correct, but I can't find the way to display it as a string in London time. @Alexander got it. – jf328 Mar 31 '15 at 10:57
  • related: [Converting timezones from pandas Timestamps](http://stackoverflow.com/q/25653529/4279) – jfs Mar 31 '15 at 22:25

1 Answers1

14

DST is relative to your location (e.g. London DST began a few weeks after NY). You first need to make the timestamp timezone aware: 

from pytz import UTC
from pytz import timezone
import datetime as dt

ts = pd.Timestamp(datetime.datetime(2015, 3, 31, 15, 47, 25, 901597))
# or...
ts = pd.Timestamp('2015-03-31 15:47:25.901597')
# ts is a Timestamp, but it has no idea where in the world it is...
>>> ts.tzinfo is None
True

# So the timestamp needs to be localized.  Assuming it was originally a UTC timestamp, it can be localized to UTC.
ts_utc = ts.tz_localize(UTC)
# Once localized, it can be expressed in other timezone regions, e.g.: 
eastern = pytz.timezone('US/Eastern')
ts_eastern = ts_utc.astimezone(eastern)
# And to convert it to an ISO string of local time (e.g. eastern):
>>> ts_eastern.isoformat()
'2015-03-30T08:09:27.143173-04:00'

See pytz or datetime for more information.

Alexander
  • 105,104
  • 32
  • 201
  • 196
  • if `dt` is the `datetime` module then use `utcnow()` instead `now()` here. – jfs Mar 31 '15 at 22:21
  • Sure, but the point is to show how to convert a local timestamp to UTC (the 'now' instance is just an example). It is considered best practice to always deal in UTC time except for when displaying. – Alexander Mar 31 '15 at 22:43
  • Europe/London timezone != UTC!!! Do not use `.now()` (returns time in *local* timezone) then you mean `.utcnow()` (returns time in UTC)! – jfs Mar 31 '15 at 22:43
  • Never said they were, but I updated the example to clarify the steps. – Alexander Mar 31 '15 at 22:58
  • Do understand that using `.now()` instead of `.utcnow()` is equivalent to saying that `Europe/London` == `UTC` in this particular case? (OPs local timezone is Europe/London). – jfs Mar 31 '15 at 23:02
  • Yes I do, but I never said the original timestamp was in either location. I was merely pointing out that time is relative, and even things like DST can be implemented at different dates in different regions. Like I said, I clarified the example to remove the .now() example in favor of a specific datestamp instance. The point is that it is needs to be localized (to UTC assuming the original stamp was recorded as UTC). Once localized, it can then converted/expressed in other regions. – Alexander Mar 31 '15 at 23:36
  • Just to say that I find quite strange that "2015-03-31 15..." UTC corresponds to "2015-03-30 08..." US eastern time: more than 24 hours of difference! There is some incoherence here I suppose... – Songio May 27 '22 at 11:19