Converting DateTime stamps into unix with Python

Question

I can't seem to get a time stamp into a unix timestamp format and assume I'm missing some points. Looking for some help.

I have a column within a df that is a datetime object in the format: YYYY-MM-DD HH:MM:SS and I need a new column with the unix stamp.

I'm bringing the csv into python with:

gps = filepath here   
dateparse= lambda x: pd.datetime.strptime(x, '%Y%m%d %H:%M:%S')
gps_dat = pd.read_csv(gps, parse_dates=['date_stamp'],date_parser=dateparse)

Anytime I try to change this column into a unix stamp I'm getting an error about wrong format or "datetime.datetime" has no attribute "datetime" with this:

gps_dat['unix']=datetime.datetime(gps_dat['date_stamp'])

Should I be using

calendar.timegm(tuple)

I'm still learning so any help would be greatly appreciated!

Answer 1

You can use faster solution with list comprehension:

print gps_dat
  nam  code date1          date_stamp
0   a     1   1/1 2012-10-08 18:15:05
1   b     3   3/4 2012-10-08 18:15:05

gps_dat['unix'] = [t.value // 10 ** 9 for t in gps_dat['date_stamp']]
print gps_dat
  nam  code date1          date_stamp        unix
0   a     1   1/1 2012-10-08 18:15:05  1349720105
1   b     3   3/4 2012-10-08 18:15:05  1349720105

Timings:

In [46]: %timeit gps_dat['date_stamp'].astype(np.int64) // 10**9
1000 loops, best of 3: 204 µs per loop

In [47]: %timeit [t.value // 10 ** 9 for t in gps_dat['date_stamp']]
The slowest run took 4.99 times longer than the fastest. This could mean that an intermediate result is being cached 
10000 loops, best of 3: 24.2 µs per loop

Answer 2

UPDATE: if you want to convert datetime column to UNIX timestamp:

gps_dat['unix']=gps_dat['date_stamp'].astype(np.int64) // 10**9

NOTE: but it must be of datetime type, not string/object

Old answer: parsing from UNIX timestamp to datetime

try to change your parser function like this:

dateparse= lambda x: pd.to_datetime(x, unit='s')

this will instruct to_datetime() that you're using UNIX timestamp format

Answer 3

Edit: I've never used Pandas, but it looks like this is the kind of function that you should call:

gps_dat['unix'] = gps_dat.apply(lambda row: time.mktime(row['date_stamp'].timetuple()), axis=col_number)

where col_number is the index of your date_stamp column (assuming it was correctly parsed as a datetime).

Original answer (when I didn't know Pandas was involved):

Replace that

gps_dat['unix']=datetime.datetime(gps_dat['date_stamp'])

line with

gps_dat['unix'] = time.mktime(gps_dat['date_stamp'].timetuple())

And add

import time

to your imports. Note that some considerations apply to time zones, so according to your requirements you might want to add some logic to, say, convert to UTC.