1

Is there a clock in Linux with nanosecond precision that is strictly increasing and maintained through a power cycle? I am attempting to store time series data in a database where each row has a unique time stamp. Do I need to use an external time source such as a GPS receiver to do this? I would like the time stamp to be in or convertible to GPS time.

This is not a duplicate of How to create a high resolution timer in Linux to measure program performance?. I am attempting to store absolute times, not calculate relative time differences. The clock must persist over a power cycle.

Community
  • 1
  • 1
Patrick
  • 147
  • 1
  • 15
  • What if two rows are inserted within the same nanosecond? – melpomene Jun 17 '16 at 23:42
  • Possible duplicate of [How to create a high resolution timer in Linux to measure program performance?](http://stackoverflow.com/questions/6749621/how-to-create-a-high-resolution-timer-in-linux-to-measure-program-performance) – Stephen P Jun 17 '16 at 23:46
  • @StephenP I don't think so. I'm looking to store absolute times, not calculate time differences. – Patrick Jun 17 '16 at 23:53
  • The issue is still _how to get a high-precision time on Linux_ so I think [that question](http://stackoverflow.com/q/3523442/17300) is relevant, but sure, I'll retract my close-vote. The problem is that `CLOCK_MONOTONIC` resets, but `CLOCK_REALTIME` can vary (skip backwards). Your best bet (barring specialized hardware) may be to get the time to only 1 second precision and append the Monotonic clock to that. – Stephen P Jun 17 '16 at 23:59
  • 2
    The precision of the clock depends on the circuits on the motherboard,which would probably be milliseconds or microseconds. Furthermore, many systems have multiple CPU's, meaning that is quite possible that multiple records will be inserted at the same time, especially if they are inserted as a group. The main issue for using GPS time is to ignore leap seconds. There are also limitations on the resolution of times in the data base. If you want resolution at better than microseconds, you are probably looking at custom hardware and software. – Bradley Ross Jun 18 '16 at 00:00
  • @melpomene I do not believe that the time between the events being recorded will be less than a nanosecond. – Patrick Jun 18 '16 at 00:01
  • 1
    Any timestamp value that is bijective with GPS time is by definition not "unique", because GPS time is common to many CPUs, many locations, etc. – Ben Voigt Jun 18 '16 at 00:06
  • @BradleyRoss The time stamp for each event is calculated when it occurs, not when it is written to the database. – Patrick Jun 18 '16 at 00:10
  • @BenVoigt I meant unique within the database. – Patrick Jun 18 '16 at 00:11
  • @StephenP That is an interesting suggestion. Do you suppose `CLOCK_MONOTONIC` and `CLOCK_REALTIME` increment at the same second? – Patrick Jun 18 '16 at 00:15
  • @Patrick - even if they don't, does it matter? The monotonic clock will always be increasing; the danger is that even a 1-second clock _might_ go backward with a time adjustment. The resulting combination would still be unique, but may not be strictly-increasing ... e.g.: `10 123, 10 124, 9 125, 10 126` – Stephen P Jun 18 '16 at 00:19
  • What do you hope to do with these timestamps finally? Graph things? Compute elapsed time between samples? Nanoseconds go by *really* fast and you're talking about a database. Are you sure you want the time the record is prepared for insertion into the database and not the time the sample was actually acquired? – Ben Voigt Jun 18 '16 at 00:25
  • @StephenP Yes, that wouldn't be good. – Patrick Jun 18 '16 at 00:26
  • @BenVoigt I am recording the values of process variables that come in over the network from various sources. The combination of the process variable name and the time it was read from the network make up the primary key in the database. My primary concern is not really with getting the absolute exact time of the event. I mostly want to make sure that no two events have the same time, and that the order of the times matches the order of the events. The events may come in within a second of each other, so the first requirement requires dealing with the application of leap seconds. – Patrick Jun 18 '16 at 00:40
  • "come in over the network"... if that's Ethernet, you've already lost the order of occurrence. Anyway, your requirement would seem to be met by a monotonic clock accurate to one second, plus a counter. Or at least that will be no worse than any other choice, considering how decoupled the time of processing will be from the actual event being detected. – Ben Voigt Jun 18 '16 at 00:50

4 Answers4

1

Most computers now have software that periodically corrects the system time using the internet. This means that the system clock can go up or down some milliseconds every so often. Remember that the computer clock has some drift. If you don't want problems with leap seconds, use sidereal time (no leap second corrections). ntp will be off in the microsecond or millisecond range because of differences in latency over the internet. The clocks that would meet your requirements are fifty thousand dollars and up.

Bradley Ross
  • 445
  • 2
  • 8
  • Do I need special hardware to get the sidereal time? – Patrick Jun 18 '16 at 00:50
  • Look at http://tycho.usno.navy.mil. It explains many of these issues. Basically, you're looking at the GPS time that doesn't include leap seconds. (This is the USA official time-keeper, the United States Naval Observatory.) It has all of the information that you could want about the various time standards: UT1, UTC, etc. Computers are driven by internal clock cycles. If two actions occur in the same clock cycle but on different cores, they will occur in the same nanosecond. How about randomly adding ten nanoseconds to one of the times if two events occur at the same nanosecond. – Bradley Ross Jun 18 '16 at 03:56
1

In no particular order:

To sure your time is free of savings time changes use date's -u argument to use UTC time. That is always increasing, baring time corrections from system admins.

The trouble with %N is that the precision you actually get depends on the hardware and can be much less than %N allows. Run a few experiments to find out. Warnings on this tend to be everywhere but it is still overlooked.

If you are writing Cish code, see the utime() function and use gmtime(), not localtime() type functions to convert to text. Look at the strftime() function to format the integer part of the time. You will find the strftime() format fields magically match those of the date command formats as basically, date calls strftime. The true paranoid willing to write additional code can use CLOCK_MONOTONIC to be sure your time is increasing.

If you truly require increasing times you may need to write your own command or function that remembers the last time. If called during the same time add and offset of 1. Keep incrementing the offset as often as required to assure unique times until you get hardware time greater than your adjusted time.

Linux tends to favor NTP to obtain Network time. The previously mentioned function to assure increasing time will help with backwards jumps as the jumps are usually not large.

Community
  • 1
  • 1
Gilbert
  • 3,740
  • 17
  • 19
1

Based on the question, the other answers, and discussion in comments...
You can get "nanosecond precision that is strictly increasing and maintained through a power cycle" by combining the results of clock_gettime() using the CLOCK_REALTIME and from using CLOCK_MONOTONIC - with some caveats.

First, turn off NTP. Run NTP once at each system restart to sync your time with the world, but do not update the time while the system is up. This will avoid rolling the time backwards. Without doing this you could get a sequence such as

20160617173556 1001
20160617173556 1009
20160617173556 1013
20160617173555 1020 (notice the second went backward)
20160617173556 1024

(For this example, I'm just using YYMMDDhhmmss followed by some fictional monotonic clock value.)

Then you face business decisions.

• How important is matching the world's time, compared to strictly increasing uniqueness?
. (hardware clock drift could throw off accuracy with world time)
• Given that decision, is it worth the investment in specialized hardware, rather than a standard (or even high-end) PC?
• If two events actually happen during the same nanosecond, is it acceptable to have duplicate entries?
• etc.

There are many tradeoffs possible based on the true requirements that lead to developing this application.

Stephen P
  • 14,422
  • 2
  • 43
  • 67
  • Does the NTP time include leap seconds? If not, is it possible to sync the system to it but not apply leap seconds to the system time? – Patrick Jun 18 '16 at 01:10
  • Perhaps I could just set the NTP daemon to slew the leap second? – Patrick Jun 18 '16 at 01:36
0

If nanosecond precision is really sufficient for you: date +%s%N

pjanek
  • 76
  • 1
  • 5
  • Is the result of this strictly increasing? Will it repeat during the application of a leap second? – Patrick Jun 17 '16 at 23:54
  • Other than forcing a *ntp* update at boot and on a regular basis (depending on the drift rate could be hours, days, weeks, ...) there isn't any machine specific clock that is guaranteed to be always increasing. Even with the *leap* second issue, that basis for time will be adjusted. You are running up against real-world limitations. – David C. Rankin Jun 18 '16 at 00:05
  • Hypothetically it's possible that two calls will be done in same nanosecond, but in practice it's very unlikely (especially on single thread). Moreover, as @Bradley Ross said, there are hardware limitations. Yes, repeat is possible in case of leap second. – pjanek Jun 18 '16 at 00:23
  • @pjanek In testing I had to move to nanoseconds because otherwise I was getting two calls with the same time. – Patrick Jun 18 '16 at 00:44