1

I am looking to provide simple logs to my users via my api from different time zones and I am a bit lost would appreciate some help.

I have the following table called logs:

CREATE TABLE `logs` (
   `id` INT(11) NOT NULL AUTO_INCREMENT,
   `userId` int(11) NOT NULL,
   `ip` VARCHAR(45) NOT NULL,
   `timestamp` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
   PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

And I have an hourly table to bind my query over 24 hours.

CREATE TABLE `log_hours` (
   `id` INT(11) NOT NULL AUTO_INCREMENT,
   `hourId` int(11) NOT NULL,
   PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO log_hours
    (hourId)
VALUES
    (1),
    (2),
    (3),
    (4),
    (5),
    (6),
    (7),
    (8),
    (9),
    (10),
    (11),
    (12),
    (13),
    (14),
    (15),
    (16),
    (17),
    (18),
    (19),
    (20),
    (21),
    (22),
    (23),
    (24);

Every time I get an api hit I store the log take the following data.

id | userId |   ip    | timestamp
1    1        0.0.0.0   2018-08-23 14:20:34
2    1        0.0.0.0   2018-08-23 14:20:34
3    1        0.0.0.0   2018-08-23 14:20:34
4    1        0.0.0.0   2018-08-23 14:20:34
5    1        0.0.0.0   2018-08-23 14:20:34

Now if I run my query like below.

SELECT DW.hourId AS hour, ifnull(t.hits,0) hits from log_hours DW left join (            
    SELECT COUNT( * ) AS hits, HOUR( logs.timestamp ) AS `hour` 
    FROM logs WHERE DAY( logs.timestamp ) = DAY(CURDATE()) 
    AND MONTH( logs.timestamp ) = MONTH(CURDATE()) 
    AND `userId` = 1 GROUP BY HOUR( logs.timestamp )) t on DW.hourId = t.hour 
    ORDER BY hour ASC

This works fine and gives me the logs for each hour like this.

enter image description here

Now I am stuck on time zones my database time zone is set to BST.

Say I have a user in America for instance if they hit the api in their time zone does mysql automatically convert their time zone to my BST time zone so the output data would make sense?

Should I be using datetime NOT NULL DEFAULT CURRENT_TIMESTAMP or should I be using an INT and using php time() function to insert the record?

Should I return the data first via php from my timezone and then convert to theirs what is best practice in this scenario?

EDIT:

First of sorry if I am asking stupid questions here but really want to fully understand the process.

At what point do I retrieve the user's local timezone? should I add it as a required param to the API.

Here is my PHP code I am just adding everything to one function makes it easier to read my process is MVC.

public function usage_get(){

        $userId = $this->get('userId'); // i am obviously not getting the users info this way this is just for the example

        // DO I MAKE THEM SEND THEIR TIMEZONE TO THE API AS A PARAM THEN USE IT IN MYSQL
        //https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_convert-tz?
        $timezone = $this->get('timezone');

        // DO I SOMEHOW RETRIEVE THE TIME ZONE BASE ON IP AND GET IT THAT WAY?

        $query = $this->db->query("SELECT DW.hourId AS hour, ifnull(t.hits,0) hits
        from log_hours DW 
        left join (

            SELECT COUNT( * ) AS hits, HOUR( logs.timestamp ) AS `hour`
            FROM logs
            WHERE DAY( logs.timestamp ) = DAY(CURDATE())
            AND MONTH( logs.timestamp ) = MONTH(CURDATE())
            AND `userId` = ".$this->db->escape_str($userId)."
            GROUP BY HOUR( logs.timestamp )

        ) t  on DW.hourId = t.hour ORDER BY hour ASC");

        // DO I RUN A PHP FUNCTION AFTER THE MYSQL BASED ON TIMEZONE?

        if($query->num_rows() > 0){

            $this->response([
                'status'  => TRUE,
                'message' => 'Success',
                'data'    => $query->result(),
            ], REST_Controller::HTTP_OK);


        }else{ 

            $this->response([
                'status' => FALSE,
                'message' => 'Error'
            ], REST_Controller::HTTP_OK);

        }

    }
Blackbam
  • 17,496
  • 26
  • 97
  • 150
user1503606
  • 3,872
  • 13
  • 44
  • 78

3 Answers3

1

Keeping it simple is always the best idea. Save all dates independent from local time. Everything else is bad practice in my eyes as local times only lead to confusion and problems later.

Say I have a user in America for instance if they hit the api in their time zone does mysql automatically convert their time zone to my BST time zone so the output data would make sense?

If you want to deliver the log time as local time convert it. In the database always UTC(+/-0) should be stored. In MySQL you can use the CONVERT_TZ function (https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_convert-tz) during the query.

Should I be using datetime NOT NULL DEFAULT CURRENT_TIMESTAMP or should I be using an INT and using php time() function to insert the record?

CURRENT_TIMESTAMP seems to be the right approach for this as it is independent (Should MySQL have its timezone set to UTC?). I can not see any good reason to use the PHP time() function because CURRENT_TIMESTAMP is most precise and always correct if the server time is set correctly, however basically there should be no difference besides the milliseconds between the call from PHP and the actual database insert.

Should I return the data first via php from my timezone and then convert to theirs what is best practice in this scenario?

In my opinion conversion using the database is best (fastest) - at least if you can write your own SQL queries. Otherwise conversion via PHP is totally legit, however there is some minimal overhead. How to do this has been asked before: Convert UTC dates to local time in PHP

At what point do I retrieve the user's local timezone? Should I add it as a required param to the API?

This is a question of application architecture, there is no "rule" for it. Possibilities to retrieve the users' timezone:

  1. GeoIP: Use GeoIP and location-based matching to match the IP to a zone. The downside here, as with many solutions, is that if they are using a VPN or Proxy, it will show the zone of the VPN or Proxy IP, not the users physical IP that's behind it.

  2. JavaScript: Javascript is client-side, though much like GeoIP, if the client is behind a VPN or Proxy, the same issue persists. You won't be displaying the timezone of the user, you'll be displaying the timezone of the VPN or Proxy (Getting the client's timezone in JavaScript).

  3. Ask the user: The timezone is passed as parameter somehow.

For an API I would definitly recommend the latter - ask for the timezone as parameter or simply pass them the time UTC (+-0). Maybe users which are using your API have a profile where they set their time zone?

If not think about passing them a result which contains the time in UTC (+-0) as well as the local time with the guessed timezone shortcode. Any implementation is valid, just make it clear for which timezone the result is and how the user queries correctly.

Blackbam
  • 17,496
  • 26
  • 97
  • 150
  • Thanks, @Blackbam I have updated my question at the bottom with my code to try and better understand the correct place to retrieve the user's timezone – user1503606 Aug 23 '18 at 14:55
  • Hey sure it is better you make it clear. Wait a minute I update my answer. – Blackbam Aug 23 '18 at 15:46
  • Hi Blackbam thanks for this really appreciate the response I passed out yesterday ;) Think I will ask the user to send their timezone as a param easiest for me and all round. – user1503606 Aug 24 '18 at 08:09
  • I have also update in question with FINAL THOUGHTS: hopefully i am on the right track – user1503606 Aug 24 '18 at 08:39
  • @user1503606 Thanks for sharing all this in detail. I made a *community wiki answer* out of it for future readers to be less confused. – Blackbam Aug 24 '18 at 15:01
0

You will have less headache if you record all timestamps in universal time. Tracking whom had which daylight savings time where gets very old very quickly.

Dave Stokes
  • 775
  • 4
  • 10
  • Thanks, Dave I was just reading this post https://stackoverflow.com/questions/2505681/timezone-conversion-in-php which also mentions UTC going to set that up now – user1503606 Aug 23 '18 at 13:59
0

FINAL THOUGHTS (my solution by questionare):

Ok, first thanks Blackbam for the info it really helped I have also been reading this article that helped.

http://www.vertabelo.com/blog/technical-articles/the-proper-way-to-handle-multiple-time-zones-in-mysql

From what I understand UTC is a reference for a starting point for the 24 hour time clock, this can then be offset to be converted into the correct timezone this used to be GMT or GMT is the same not sure but it was a starting point.

When the original time reference had to be defined, it was agreed to make it relative to a certain point on earth. Greenwich, a borough of London that is home to the prime meridian (0o longitude) was chosen as this point.

The offset is then represented by functions whether in PHP or MySql that work out the offset based on string params, for instance, these can change and are stored in a global database.

MySQL:

SELECT CONVERT_TZ('2004-01-01 12:00:00','GMT','MET');

PHP:

date_default_timezone_set('America/Los_Angeles');

After vaguely understanding this I can now see a solution for my setup because I will have multiple users from different timezones sign up for my service it makes sense to me to have an option under their account profile to set their correct timezone.

This then enables me to grab their time zone and then update the mysql query to return the correct data in their timezone I agree with Blackbam I would prefer using mysql.

My process is as follows.

  1. Mysql database storing time or setting to UTC (still looking for the correct way to do this.)

**Update: **

I think I have done this with the following commands log into mysql and at the command line run.

SELECT @@global.time_zone, @@session.time_zone;

This then returns your time zone mine was.

mysql> SELECT @@global.time_zone, @@session.time_zone;
+--------------------+---------------------+
| @@global.time_zone | @@session.time_zone |
+--------------------+---------------------+
| SYSTEM             | SYSTEM              |
+--------------------+---------------------+
1 row in set (0.00 sec)

I then ran.

mysql> SET GLOBAL time_zone = '+00:00';

And now I get.

mysql> SELECT @@global.time_zone, @@session.time_zone;
+--------------------+---------------------+
| @@global.time_zone | @@session.time_zone |
+--------------------+---------------------+
| +00:00             | SYSTEM              |
+--------------------+---------------------+
1 row in set (0.00 sec) 
  1. Users set their timezone through their account the same way you would set a timezone in a blogging platform say Wordpress for instance.

  2. The timezone is stored in the database and then retrieved via the API for conversion.

Like this for instance.

$timezone = $this->get('timezone');
$query = $this->db->query("SELECT DW.hourId AS hour, ifnull(t.hits,0) hits
from log_hours DW 
left join (

    SELECT COUNT( * ) AS hits, HOUR( CONVERT_TZ(logs.timestamp,'GMT','".$this->db->escape_str($timezone)."') ) AS `hour`
    FROM logs
    WHERE DAY( CONVERT_TZ(logs.timestamp,'GMT','".$this->db->escape_str($timezone)."') ) = DAY(CURDATE())
    AND MONTH( CONVERT_TZ(logs.timestamp,'GMT','".$this->db->escape_str($timezone)."') ) = MONTH(CURDATE())
    AND `userId` = ".$this->db->escape_str($userId)."
    GROUP BY HOUR( CONVERT_TZ(logs.timestamp,'GMT','".$this->db->escape_str($timezone)."') )

) t  on DW.hourId = t.hour ORDER BY hour ASC");

I am currently getting NULL from CONVERT_TZ so also just figuring out the best way to solve this apparently you need to load the zoneinfo into mysql.

This fixed my timezone issue:

./mysql_tzinfo_to_sql /usr/share/zoneinfo | ./mysql -uroot -proot mysql

This means I can now run.

SELECT CONVERT_TZ(NOW(), @@session.time_zone, 'America/Los_Angeles');
SELECT CONVERT_TZ(NOW(), @@session.time_zone, 'Indian/Antananarivo');
SELECT CONVERT_TZ(NOW(), @@session.time_zone, 'Asia/Dhaka');
SELECT CONVERT_TZ(NOW(), @@session.time_zone, 'Antarctica/Troll');

output:

CONVERT_TZ(NOW(), @@session.time_zone, 'America/Los_Angeles')
2018-08-24 02:07:36

CONVERT_TZ(NOW(), @@session.time_zone, 'Indian/Antananarivo')
2018-08-24 12:07:36

CONVERT_TZ(NOW(), @@session.time_zone, 'Asia/Dhaka')
2018-08-24 15:07:36

CONVERT_TZ(NOW(), @@session.time_zone, 'Antarctica/Troll')
2018-08-24 11:07:36

You can get the timezones from here: http://php.net/manual/en/timezones.php

Getting there please let me know if I am getting this all wrong.

Blackbam
  • 17,496
  • 26
  • 97
  • 150