Converting timestamp to unix date python

Question

Here is how the timestamp looks -

2015-07-17 06:01:51.066141+00:00

I'm looking around to convert this to unix date time.

datetime.strptime("2015-07-17 06:01:51.066141+00:00", "%Y-%m-%d %H:%M:%S.%f%z").strftime("%s")

ValueError: 'z' is a bad directive in format '%Y-%m-%d %H:%M:%S.%f%z'

throws error for me, probably because of wrong format being used.

PS: my virtualenv is on python 2.7

ideas please ?

https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior — Konstantin, Jul 21 '15 at 06:39
I gave that the try - getting - ValueError: time data '2015-07-17 06:01:51.066141+00:00' does not match format '%Y-%m-%d %H:%M:%S.%f+%Z' — user2119554, Jul 21 '15 at 06:58
Python does not support `.strftime("%s")`, see http://stackoverflow.com/a/11743262/4663466 or http://stackoverflow.com/a/31096353/4663466 — Scott, Jul 21 '15 at 07:02
Yes, the `strptime` does not seem to support the "+hh:mm" style UTC offset. I think you should consider looking into the source of the timestamp string: Where does it come from? Can you make it have some other format? Are you guaranteed this format? — skyking, Jul 21 '15 at 07:03
updated question with error message - this is what I am getting from models.DateTimeField — user2119554, Jul 21 '15 at 07:08
possible duplicate of [Convert python datetime to epoch with strftime](http://stackoverflow.com/questions/11743019/convert-python-datetime-to-epoch-with-strftime) — Anand S Kumar, Jul 21 '15 at 07:09
please accept one of the many answers here so future readers know what worked. — GG_Python, Jul 21 '15 at 16:18

GG_Python · Answer 1 · 2015-07-22T17:27:14.240

2

For Python 2.7 use arrow:

import arrow    
date_str = "2015-07-17 06:01:51.066141+00:00"
unix_time = arrow.get(date_str).timestamp

On PY3 (verified on 3.4), using only standard libs

The date string you show will not be parsed by the standard python datetime library since it has a colon in the timezone (see here). The colon can be easily removed since it's always in the same position (or use rfind to find its index starting from the right). Your simplest solution is:

import datetime

date_str = "2015-07-17 06:01:51.066141+00:00"

date_str_no_colon = date_str[:-3]+date_str[-2:] # remove last colon
dt_obj = datetime.datetime.strptime(date_str_no_colon, "%Y-%m-%d %H:%M:%S.%f%z")
unix_time = dt_obj.timestamp()

Note that arrow should still work with PY3, and is a better solution in general- you don't want to get into datetime parsing wars with python. It will win.

edited Jul 22 '15 at 17:27

answered Jul 21 '15 at 07:21

GG_Python

3,436
5
34
46

throws same error - ValueError: 'z' is a bad directive in format '%Y-%m-%d %H:%M:%S.%f%z' – user2119554 Jul 21 '15 at 07:23
interesting! I'm working on python 3.4, works fine. And you're right, I verified it doesn't work with 2.7.. – GG_Python Jul 21 '15 at 07:25
This solution relies on newer version of python, however I think it's a step in the right direction. When using `datetime` to parse the timestamp you will have to make a similar workaround for the timezone offset. You could for example use `datetime` to parse the UTC offset and the rest of the timestamp separately and then combine the results. – skyking Jul 21 '15 at 07:26
1

[arrow](https://github.com/crsmithdev/arrow) seems to have the functionality you'll need without having to parse yourself. – GG_Python Jul 21 '15 at 07:28
note: `arrow.timestamp` drops fractions of a second (it returns an integer timestamp). – jfs Jul 22 '15 at 17:43

score 2 · Answer 2 · answered Jul 21 '15 at 07:34

2

python 2.7 strptime() does not support z directive, either you can use python 3.2+ or some other 3rd party library like dateutil.

answered Jul 21 '15 at 07:34

NIlesh Sharma

5,445
6
36
53

note: `%z` does not understand the colon in the utc offset even in Python 3.2+, see [Convert timestamps with offset to datetime obj using `strptime`](http://stackoverflow.com/q/12281975/4279) – jfs Jul 22 '15 at 17:30

score 1 · Answer 3 · edited May 23 '17 at 12:21

1

strptime() has no support for timezones. So, you can make the conversion ignoring the timezone in the following way:

datetime.strptime("2015-07-17 06:01:51.066141", "%Y-%m-%d %I:%M:%S.%f").strftime("%s")
'1437102111'

Or in order to avoid using %s, as suggested below in the commments :

from datetime import datetime
(datetime.strptime("2015-07-17 06:01:51.066141", "%Y-%m-%d %I:%M:%S.%f") - datetime(1970, 1, 1)).total_seconds()
1437112911.066141

Notice this is a working version for Python 2, you can also check solutions for other versions here

Otherwise, you will have to use other libraries (django.utils or email.utils) that support timezones, or implement the timezone parsing on your own.

P.S. :

strptime docs appear to have support for timezone, but in fact it has not been implemented. Try :

datetime.strptime("2015-07-17 06:01:51.066141+00:00", "%Y-%m-%d %I:%M:%S.%f%z").strftime("%s")

and you will see that it is not supported. You can also verify it by searching more about strptime()

edited May 23 '17 at 12:21

Community

1
1

answered Jul 21 '15 at 07:06

Dimos

8,330
1
38
37

Wrong, datetime has support for timezones like `+0530` (without the **:** colon in between) , use the `%z` for that. – Anand S Kumar Jul 21 '15 at 07:07
1

And python does not support the `%s` symbol, that is an artifact of the underlying c's `strftime()` function. And that only works in linux. – Anand S Kumar Jul 21 '15 at 07:08
I would not recommend to people to use `.strftime("%s")` http://stackoverflow.com/a/11743262/4663466 – Scott Jul 21 '15 at 07:09
@AnandSKumar Well it kind of supports timezones that way, however `strptime` does not seem to accept `%z`. – skyking Jul 21 '15 at 07:10
yes it does , like I said without the colon in the middle. – Anand S Kumar Jul 21 '15 at 07:11
Example - `datetime.strptime("2015-07-17 06:01:51.066141+0000", "%Y-%m-%d %H:%M:%S.%f%z")` . Try this. – Anand S Kumar Jul 21 '15 at 07:11
@AnandSKumar With a `%z` my python gives me: ValueError: 'z' is a bad directive in format '%Y-%m-%d %H:%M:%S.%f %z'. Does not seem to work. Try it yourself... – skyking Jul 21 '15 at 07:14
@skyking what version are you using? – Anand S Kumar Jul 21 '15 at 07:16
@AnandSKumar 2.7.3. In python 3.2.3 `%z` it seem to work, but according to the updated question the OP does not seem to use that. – skyking Jul 21 '15 at 07:17
@skyking yes, i tried it in PYthon 2.x , seems like its not implemented there, it has been implemented in Python 3 though. – Anand S Kumar Jul 21 '15 at 07:22
`strftime('%s')` is incorrect for utc input (look at the result) even if your system supports %s – jfs Jul 23 '15 at 06:58

score 1 · Answer 4 · edited May 23 '17 at 11:51

1

The way to parse the date is not right. You'll either need to parse it by hand, find another library (for example the dateutil.parser.parse method that will parse your string directly without supplying format string) that supports that format or make the timestamp in another format. Even with newer versions of python the %z directive does not accept TZ offsets in the +/-HH:MM format (with colon).

As the source of the timestamp is django.DateTimeField maybe this question can help?

For converting to unix timestamp you seem to have to do some work since there does not seem to be a direct method for that:

(t - datetime.utcfromtimestamp(0)).total_seconds()

where t is the datetime (assuming it's in UTC and there is no tzinfo) you want to convert to POSIX timestamp. If the assumption is not correct you need to put tzinfo in the zero timestamp you subtract as shown below where the assumption does not hold.

If you want to use dateutil.parser the complete solution would be:

(dateutil.parser.parse(timestamp) - datetime.utcfromtimestamp(0).replace(tzinfo=utc()).total_seconds()

edited May 23 '17 at 11:51

Community

1
1

answered Jul 21 '15 at 07:13

skyking

13,817
1
35
57

while creating the datetime object, I'm still facing issue - datetime.datetime.strptime("2015-07-17 06:01:51.066141+00:00", "%Y-%m-%d %H:%M:%S.%f%z") ValueError: 'z' is a bad directive in format '%Y-%m-%d %H:%M:%S.%f%z' – user2119554 Jul 21 '15 at 07:27
@user2119554 Yes I know that, you have to use one of the three alternatives for parsing the timestamp that I mentioned (directly using `datetime` module won't work). Where do you get the timestamp from? – skyking Jul 21 '15 at 07:28
its created using models.DateTimeField – user2119554 Jul 21 '15 at 07:29
@user2119554 Are you meaning django? Are you sure that the timestamp will be formatted in that way? Maybe `arrow` (mentioned in GG_Python's answer may help you parsing the timestamp anyway? – skyking Jul 21 '15 at 07:36
yeh django, I gone through through it. looks good, However I am not sure which method would help me out converting to unix date. - t = arrow.get('2015-07-17 06:01:51.066141+00:00') – user2119554 Jul 21 '15 at 07:38
Also there's a possibility that the `DateTimeField` can convert to `datetime` directly. The documentation it mentions "Normalizes to: A Python datetime.datetime object." whatever that means. – skyking Jul 21 '15 at 07:40
@J.F.Sebastian Thanks for pointing that out, I've updated the answer to use `utcfromtimestamp` instead. – skyking Jul 23 '15 at 05:52
`dateutil.parser` may return an aware datetime that won't work with the naive `utcfromtimestamp(0)`. You could make the latter aware using `.replace(tzinfo=utc)` where `utc` is a [simple tzinfo subclass](http://stackoverflow.com/a/25421145/4279) – jfs Jul 23 '15 at 05:56
@J.F.Sebastian Yeah, I had some doubts about that `t` could have wrong timezone. Hovewer replacing it is probably not the right thing to do as it just removes the information and consequently does not result in a `datetime` representing the same instant, I updated it to use `astimezone` instead. – skyking Jul 23 '15 at 06:15
It is wrong. My previous comment is about the 2nd formula, not the 1st one (where do you see dateutil there?) Revert the 1st formula and use `utcfromtimestamp(0).replace(tzinfo=utc)` or `datetime(1970, 1, 1, tzinfo=utc)` in the 2nd one. `astimezone()` usage is incorrect and you don't need it whatever input utc offset is (it is handled automatically in the 2nd case) – jfs Jul 23 '15 at 06:21
@J.F.Sebastian Well, yes and no. The method of modifying the first term works as well, but your suggestion is more elegant. In the first snippet theres actually no assumption about `t` (apart from that it's a `datetime` object) so you cannot actually tell without that whether you should add `tzinfo` to the second term or not. Updated the answer to reflect both these observations. – skyking Jul 23 '15 at 06:38

score 0 · Answer 5 · edited May 23 '17 at 10:26

There are two parts:

to convert "2015-07-17 06:01:51.066141+00:00" into a datetime object that represents UTC time, see Convert timestamps with offset to datetime obj using strptime. Or If you know that the utc offset is always +0000:
```
from datetime import datetime

utc_time = datetime.strptime(time_string, "%Y-%m-%d %H:%M:%S.%f+00:00")
```
to convert the UTC time to POSIX timestamp (unix time), see Converting datetime.date to UTC timestamp in Python:
```
from datetime import datetime

timestamp = (utc_time - datetime(1970, 1, 1)).total_seconds()
```

Converting timestamp to unix date python

5 Answers5