Facebook event start_time sometimes in user time, others in local event time

Question

I am trying to convert the Facebook start_time of time. As per Facebook's documentation, Facebook uses ISO-8601 strings to describe precise time with offset from UTC of format:

%Y-%m-%dT%H:%M:%S+OFFSET

On extracting from the API, some events come out as local time e.g. example 1, and others come out as my user's time e.g. example 2 (note that the location of event 1 and 2 is the same).

example 1) eid = 163113227202707, start_time = 2013-06-18T22:00:00-0700

example 2) eid = 590809384285866, start_time = 2013-06-17T01:00:00-0300

I need to use local event times including for daylight savings.

I went about creating a table for about 1500 cities of the format:

city    timezone    dst offset
Abidjan GMT     +0000
Abuja   WAT     +0100
Accra   GMT     +0000
Adama   EAT     +0300
Addis Ababa EAT     +0300
Agadez  WAT     +0100
Agadir  WEST    DST +0100
Al Jizah    EET     +0200
Alexandria  EET     +0200
Algiers CET     +0100
Ali Sabieh  EAT     +0300
Antananarivo    EAT     +0300
Antsirabe   EAT     +0300
Arusha  EAT     +0300
Asmara  EAT     +0300
Bafatá  GMT     +0000
Bamako  GMT     +0000
Bambari WAT     +0100
Bamenda WAT     +0100
Bangui  WAT     +0100
Banjul  GMT     +0000
Bata    WAT     +0100
Beira   CAT     +0200
Benghazi    CST DST +0200
Benin City  WAT     +0100
Berbérati   WAT     +0100

However, since this is static it doesn't help with daylight savings.

On reading the document about pytz I'm not sure how to query the timezone based on the city name. Is this even possible?

Essentially, I am trying to create the following pseudo code:

start_time = 2013-06-17T01:00:00-0300

UTC_start_time = normalize_time_to_UTC(start_time)

offset = lookup_offset_given_city_and_UTC(city, UTC)

local_start_time(UTC_start_time, offset)

Update:

I first normalize the time based on the offset provided by Facebook by parsing the string as follows:

def fql_local_time_to_utc(start_time):

    if len(fql_timestamp) < 20:
        return fql_timestamp

    else:
        timestamp = fql_timestamp[:19]
        sign = fql_timestamp[19]
        hours = fql_timestamp[20:22]
        minutes = fql_timestamp[22:24]
        offset = int(hours)+int(minutes)/60

        if sign =="+":
            offset = -offset

        timestamp_object = datetime.datetime.strptime(timestamp, '%Y-%m-%dT%H:%M:%S')
        utc_object = timestamp_object + datetime.timedelta(hours = offset)

        return utc_object 

Next, I look up the location_city of the venue_id and try to match it in the above database to which I manually added a column called "pytz_timezone_name". After finding the relevant timezone via a join query, I apply the following code.

try:           
    tz = db.fetchone()["pytz_timezone_name"]
    tz_object = timezone(tz)
    local_event_time = timezone('UTC').localize(utc_time).astimezone(tz_object)
    local_time = local_event_time.strftime('%Y-%m-%dT%H:%M:%S%z')

except:

    local_time = start_time

Answer 1

As you pointed out, you can't just have a table of cities corresponding to offsets, because of daylight savings time. Besides, offsets and time zone rules change all the time. You really need a true time zone database, such as the IANA/Olson database that pytz implements in Python.

It's impossible to convert from an offset like -07:00 to a time zone like America/Denver. There are many time zones that share the same offsets, and a single time zone can have multiple different offsets depending on the specific date/time.

Facebook gave you a date/time/offset, which you can get the precise UTC moment for. But the date/time portion of that should have been already adjusted for the local time of the event. The offset tells you by how much it has been adjusted. This adjustment should take into account DST if it applies.

However, you said that you are getting two different offsets for the same event? This sounds strange. Have you enabled the Events Timezone setting in your facebook api? This is found in the advanced settings of your Facebook app dashboard.

Regarding the pseudocode, yes that is all doable. Perhaps you want the event time adjusted for some other time zone, then this is exactly what you would do. Just be aware that it's not just a "city". That is one of the IANA named time zone identifiers. This article on Wikipedia explains it quite well.

UPDATE

In regards to the update you posted, showing your solution, I can offer the following feedback based on my vast experiences with date and time issues:

• Unless you are actually writing a library focused on parsing, there is almost never a good reason to pull out parts of a timestamp as substrings. You should parse it as a whole and use the various properties and functions that are provided by the language or framework.

• Using strings as an intermediate step is usually not required, and can be a performance bottleneck. In some languages, it can introduce nuanced error due to culture/region or local time zone settings. You started with a datetime object and you want to produce a datetime object, so all of the manipulation should be done with datetime objects and related functions.

• Python in particular uses the concept of "naive" or "aware" values of datetime. Your code starts with an "aware" value, but you are constructing a "naive" one when you call strptime. Since the input value was "aware", you should probably make sure the output value is as well. For more on this, read the first few paragraphs of these docs.

• You keep saying you are looking up cities to resolve time zones, but you haven't mentioned at all what the source of this data is. Are you manually constructing a table of cities and time zones? That approach may work during your initial development and testing, but you will probably find over time that it needs a lot of maintenance. There are lots of cities in the world. They change names sometimes, and sometimes even change their time zones. You will have to deal with misspellings, alternate spellings, naming conflicts, and more. It's probably not a good idea to do this on your own.

• There are more than a few places where a city might have multiple time zones. If you are going to try to resolve a time zone from a location, you are going to need a latitude and longitude, and one of these mechanisms.

• FYI - Python doesn't have its own set of time zone information. The pytz implements a public time zone database that has been maintained for many years by countless researchers, often called the "Olson" database, after one of its founders. This is currently coordinated by a IANA, and has it's home here. You can read more on Wikipedia, or in the timezone tag wiki here on StackOverflow.

• This data changes all the time. The current release is the third one this year (2013c). In 2012, there were 10 releases. If you want to stay current, you need to keep your pytz library up to date.

• I still don't think you need any of this. The event time coming from Facebook should already be in the correct local time and show the offset of that local time. If you are seeing the wrong offset applied, then probably the source data is wrong to begin with. Possibly the event location is wrong. There's an expression, "garbage in, garbage out". You may want to focus more on ensuring that the source data is correct, rather than trying to "fix" what you see as broken output coming from Facebook.