5

I have a text file with many rows of data - the first piece of data in each row is a unix timestamp such as 1436472000. I am using numpy.loadtxt and in the parameters for converters I want to specify for it to convert the timestamp into whatever numpy understands as a date time. I know this needs to go after the 0: in the curly brackets, but I can't work out how to convert it. I know a converter can be used from matplotlib.dates.strpdate2num for normal dates, but I this won't work for unix timestamps.

Code:

timestamp, closep, highp, lowp, openp, volume = np.loadtxt(fileName,delimiter=",",unpack=True,converters={ 0: })

Thanks for help in advance, please ask if you would like me to clarify what I mean.

user5036748
  • 55
  • 1
  • 2
  • 5

1 Answers1

9

While converters can be convenient, they are slow because they are called once for each row of data. It is faster to convert the data after the timestamps are loaded into a NumPy array of integers:

x = np.array([1436472000, 1436472001])
x = np.asarray(x, dtype='datetime64[s]')

yields an array of NumPy datetime64s:

array(['2015-07-09T16:00:00-0400', '2015-07-09T16:00:01-0400'],
       dtype='datetime64[s]')

To obtain Python datetime.datetimes use tolist():

>>> x.tolist()
# [datetime.datetime(2015, 7, 9, 20, 0), 
#  datetime.datetime(2015, 7, 9, 20, 0, 1)]

As you know, matplotlib datenums count the number of days since 0001-01-01 00:00:00 UTC, plus one. These are not timestamps (which count seconds since the Epoch, 1970-01-01 00:00:00 UTC):

>>> matplotlib.dates.date2num(x.tolist())
# array([ 735788.83333333,  735788.83334491])
unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677