Going from twitter date to Python datetime date

Question

I am receiving twitter messages that are sent at a certain date in the following format from twitter:

Tue Mar 29 08:11:25 +0000 2011

I want to store these dates in 'timestamp with time zone' field in postgresql with djangos DateTimeField field. When I store that string however I get this error:

ValidationError: [u'Enter a valid date/time in YYYY-MM-DD HH:MM[:ss[.uuuuuu]] format.']

can I automatically transform the twitter datetype to a python datetime time (that does work elsewhere in my app for saving dates).

score 97 · Accepted Answer · edited Jun 17 '20 at 17:07

97

Writing something like this should convert a twitter date to a timestamp.

import time

ts = time.strftime('%Y-%m-%d %H:%M:%S', time.strptime(tweet['created_at'],'%a %b %d %H:%M:%S +0000 %Y'))

UPDATE

For Python 3, as per 2020, you can do it in this way:

from datetime import datetime

# dtime = tweet['created_at']
dtime = 'Fri Oct 09 10:01:41 +0000 2015'
new_datetime = datetime.strftime(datetime.strptime(dtime,'%a %b %d %H:%M:%S +0000 %Y'), '%Y-%m-%d %H:%M:%S')
print((new_datetime))

edited Jun 17 '20 at 17:07

Angus

3,680
1
12
27

answered Oct 10 '11 at 11:04

Chris Herring

1,412
9
10

1

This gives me the following error, 'expected string or buffer'. Can you help me why? – Zaira Zafar Apr 08 '18 at 11:04
@ZairaZafar I added an answer below which may help. I had similar errors that you had. – Bob Aleena Jun 22 '19 at 19:25

score 15 · Answer 2 · answered Oct 09 '11 at 14:39

Give this a go. It assumes the date format from twitter is RFC822 compliant (see the question linked to by @Adrien).

A naive datetime object is constructed (i.e. no timezone info). It is adjusted according to the timezone offset to UTC. Unless you have a need to keep the original timezone, I'd store the date time as UTC and format to local time when you display it.

from datetime import datetime, timedelta
from email.utils import parsedate_tz

s = 'Tue Mar 29 08:11:25 +0000 2011'

def to_datetime(datestring):
    time_tuple = parsedate_tz(datestring.strip())
    dt = datetime(*time_tuple[:6])
    return dt - timedelta(seconds=time_tuple[-1])

I prefer this method because it even works with timezones different from +0000 (despite the fact that Twitter always uses +0000). Django side is also better to make the resulting datetime timezone-aware using the utc timezone ( see [this question](http://stackoverflow.com/questions/7065164/how-to-make-an-unaware-datetime-timezone-aware-in-python) ) — caipirginka, Apr 23 '15 at 14:13

score 8 · Answer 3 · answered May 24 '19 at 00:07

8

A little bit old but using parse really help me with this issue

from datetime import datetime
from dateutil.parser import parse

date = 'Fri May 10 00:44:04 +0000 2019' 
dt = parse(date)

print(dt) 
# 2019-05-10 00:44:04+00:00

answered May 24 '19 at 00:07

strider

683
9
25

1

This seems the best answer in 2019. Any downsides to doing it this way? – Robert Lugg Jul 25 '19 at 02:26
1

Nope. This is STILL the most pythonic approach in 2021. Use a common parser. – Marc Maxmeister Jun 30 '21 at 03:42

score 3 · Answer 4 · answered Jun 06 '19 at 13:56

To get datetime with timezone you can simple use datetime.strptime as follow:

from datetime import datetime
s = 'Wed Jun 05 05:34:02 +0000 2019'
created_at = datetime.strptime(s, '%a %b %d %H:%M:%S %z %Y')
print(created_at)
#2019-06-05 05:34:02+00:00

score 3 · Answer 5 · answered Dec 16 '20 at 02:50

Twitter API V2 sends date strings that look like this:

2020-12-15T20:17:10.000Z

This worked, to convert from string to datetime:

datetime.datetime.strptime(THE_STRING,"%Y-%m-%dT%H:%M:%S.%fZ")

The end looks like a timezone, but it's milliseconds, hence the %f. The final character, "Z" is a timezone code that means UTC, as explained, here.

score 2 · Answer 6 · edited May 23 '17 at 11:55

you can convert the date using datetime.strptime(), or time.strptime(). however, those two functions cannot parse the timezone offset (see this bug).

so, the only solution i see is to split the date yourself, remove the timezone offset, feed the rest to strptime(), and process the offset manually...

have a look at this question, where you will find some hints on how to parse the offset yourself.

score 1 · Answer 7 · answered May 22 '17 at 16:03

The following code will print a nice date (local time) from a Twitter date (UTC).

from datetime import datetime
from datetime import timezone    

datetime.strptime(mydata["created_at"], '%a %b %d %H:%M:%S %z %Y').replace(
            tzinfo=timezone.utc).astimezone(tz=None).strftime('%Y-%m-%d %H:%M:%S'))

score 1 · Answer 8 · answered Jun 22 '19 at 19:24

The initial problem I was having was converting from the datetime that the twitter api gives to String.

The following works which addresses different comments people seem to have for above solutions which are a little unclear as to whether the starting date is already in string format or not. This works for Python 2.7

With a tweet from the API, tweet.created_at gives the date in datetime format. At the top of your file, add from datetime import datetime

then use the following to get the corresponding string.

datetime.strftime(tweet.created_at,'%a %b %d %H:%M:%S %z %Y').

You can then use this string as described in other comments to manipulate it.

1man · Answer 9 · 2018-04-11T14:52:02.063

0

Using a similar strategy as SoFolichon proposed, in Python 3.x you can also use pytz like:

from datetime import datetime, timezone
import pytz

datetime.strptime(tweets["created_at"], '%a %b %d %H:%M:%S %z %Y').replace(
tzinfo=timezone.utc).astimezone(pytz.timezone('US/Eastern')).strftime(
'%Y-%m-%d %H:%M:%S')

edited Apr 11 '18 at 14:52

answered Mar 25 '18 at 15:15

1man

5,216
7
42
56

1

Hey. this gives me the following error: strptime() argument 1 must be string, not Series. – Zaira Zafar Apr 08 '18 at 11:18
@ZairaZafar, Which version of Python do you use? In Python 3.x, it works fine for me. – 1man Apr 08 '18 at 15:44
1

im using 2.7 python – Zaira Zafar Apr 10 '18 at 06:31

score 0 · Answer 10 · answered Sep 08 '18 at 11:20

How about this? It doesn't need any formatting strings.

import datetime
from email.utils import mktime_tz, parsedate_tz

def parse_datetime(value):
    time_tuple = parsedate_tz(value)
    timestamp = mktime_tz(time_tuple)

    return datetime.datetime.fromtimestamp(timestamp)

print(parse_datetime('Tue Mar 29 08:11:25 +0000 2011'))
#2011-03-29 10:11:25

My system is at GMT +2 hence the difference included.

Going from twitter date to Python datetime date

10 Answers10

Linked