4

Note: I think datetime64 is doing the right thing. So I'll just leave the post up in case it's useful.

As of numpy 1.7.0, seconds passed in to a np.datetime64 are interpreted as being in a local timezone. Is there a clean and fast way to import a unix utc seconds to np.datetime64? I've got arrays with 50M of these and it seems that there should be a way to tell np.datetime64 that my seconds value is UTC, no?

datetime.datetime.utcfromtimestamp(1338624706)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # this is the time I'm looking for

np.datetime64(1338624706, 's')
numpy.datetime64('2012-06-02T01:11:46-0700')  # Darn you ISO!  Off by 7 hours

dt64 = np.datetime64(1338624706, 's')
dt64.astype(datetime.datetime)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # Wait, did it do the right thing?

# This seems like the best option at the moment,
# but requires building datetime.datetime objects:
dt64 = np.datetime64(datetime.datetime.utcfromtimestamp(1338624706))
numpy.datetime64('2012-06-02T01:11:46.000000-0700') # Show this
dt64.astype(datetime.datetime)
datetime.datetime(2012, 6, 2, 8, 11, 46)  # Looks like it worked

I really do not want to resort to string operations. I would be nice to be able to convert an array of unix utc ints or floats straight to the correct dt64.

https://stackoverflow.com/a/13704307/417578 implies that numpy 1.8.0 might do what I want, but is there something that will work in 1.7.0?

Community
  • 1
  • 1
Kurt Schwehr
  • 2,638
  • 3
  • 24
  • 41
  • 1
    "So I'll just leave the post up in case it's useful." --- you should post the solution ('best option' I guess) as an answer so it shows up as answered. – askewchan Feb 24 '13 at 16:54
  • possible duplicate of [Converting between datetime, Timestamp and datetime64](http://stackoverflow.com/questions/13703720/converting-between-datetime-timestamp-and-datetime64) – Andy Hayden Feb 24 '13 at 18:57

2 Answers2

4

Heres another way in pandas (which handles the quirks in different versions of numpy datetime64 correctly, so this works in numpy 1.6.2) - I think u might need current master for this (0.11-dev)

# obviously replace this by your utc seconds
# need to convert to the default in pandas of datetime64[ns]
z = pd.Series([(1338624706 + i)*1e9 for i in range(50)],dtype='datetime64[ns]')

In [35]: z.head()
Out[35]: 
0   2012-06-02 08:11:46
1   2012-06-02 08:11:47
2   2012-06-02 08:11:48
3   2012-06-02 08:11:49
4   2012-06-02 08:11:50
Dtype: datetime64[ns]

# turn it into a DatetimeIndex and localize
lidx = pd.DatetimeIndex(z).tz_localize('UTC')

<class 'pandas.tseries.index.DatetimeIndex'>
[2012-06-02 08:11:46, ..., 2012-06-02 08:12:35]
Length: 50, Freq: None, Timezone: UTC

# now you have a nice object to say convert timezones
In [44]: lidx.tz_convert('US/Eastern')
Out[44]: 
<class 'pandas.tseries.index.DatetimeIndex'>
[2012-06-02 04:11:46, ..., 2012-06-02 04:12:35]
Length: 50, Freq: None, Timezone: US/Eastern
Jeff
  • 125,376
  • 21
  • 220
  • 187
2

Perhaps I misunderstand the question, but isn't the timezone just a display issue?

utc_time = datetime.datetime.utcnow()
print utc_time
dt64 =  np.datetime64(utc_time)
print dt64
print dt64.astype(datetime.datetime)


2013-02-24 17:30:53.586297
2013-02-24T11:30:53.586297-0600
2013-02-24 17:30:53.586297

The time hasn't been 'changed' in any way:

some_time = datetime.datetime.utcfromtimestamp(1338624706)
dt64 = np.datetime64(1338624706,'s')
print dt64.astype(int64)
1338624706

This is as of numpy 1.7.

radikalus
  • 555
  • 5
  • 15
  • Yes, I was just miss-reading the output. My off by 7 comment is on a line that is showing "-0700". Setting the timezone makes life easier. e.g. os.environ['TZ'] = 'UTC' – Kurt Schwehr Feb 25 '13 at 01:00