R xts: .001 millisecond in index

Question

It looks like POSIXlt allows millisecond precision specification, but I have a problem when setting a .001 millisecond index value in an xts object:

> options(digits.secs = 3)
> data(sample_matrix)
> sample.xts = xts(sample_matrix, rep(as.POSIXlt("2012-03-20 09:02:50.001"), 180))
> head(sample.xts, 10)
                            Open     High      Low    Close
2012-03-20 09:02:50.000 50.03978 50.11778 49.95041 50.11778
2012-03-20 09:02:50.000 50.23050 50.42188 50.23050 50.39767
2012-03-20 09:02:50.000 50.42096 50.42096 50.26414 50.33236
2012-03-20 09:02:50.000 50.37347 50.37347 50.22103 50.33459
2012-03-20 09:02:50.000 50.24433 50.24433 50.11121 50.18112
2012-03-20 09:02:50.000 50.13211 50.21561 49.99185 49.99185
2012-03-20 09:02:50.000 50.03555 50.10363 49.96971 49.98806
2012-03-20 09:02:50.000 49.99489 49.99489 49.80454 49.91333
2012-03-20 09:02:50.000 49.91228 50.13053 49.91228 49.97246
2012-03-20 09:02:50.000 49.88529 50.23910 49.88529 50.23910
> sample.xts = xts(sample_matrix, rep(as.POSIXlt("2012-03-20 09:02:50.002"), 180))
> head(sample.xts, 10)
                            Open     High      Low    Close
2012-03-20 09:02:50.002 50.03978 50.11778 49.95041 50.11778
2012-03-20 09:02:50.002 50.23050 50.42188 50.23050 50.39767
2012-03-20 09:02:50.002 50.42096 50.42096 50.26414 50.33236
2012-03-20 09:02:50.002 50.37347 50.37347 50.22103 50.33459
2012-03-20 09:02:50.002 50.24433 50.24433 50.11121 50.18112
2012-03-20 09:02:50.002 50.13211 50.21561 49.99185 49.99185
2012-03-20 09:02:50.002 50.03555 50.10363 49.96971 49.98806
2012-03-20 09:02:50.002 49.99489 49.99489 49.80454 49.91333
2012-03-20 09:02:50.002 49.91228 50.13053 49.91228 49.97246
2012-03-20 09:02:50.002 49.88529 50.23910 49.88529 50.23910

Why the 001 millisecond setting fails?

Answer 1

The POSIXct representation is a very clever 'hack' by Prof Ripley who breaks a standard two-byte word into 'still enough' representation for the usual 'number of days since the epoch', plus 'a suitable amount of precision for subsecond data'. It works out to around a microsecond:

R> now <- Sys.time()
R> for (x in seq(1,10)) print(difftime(now, now + 10^-x))
Time difference of -0.0999999 secs
Time difference of -0.00999999 secs
Time difference of -0.000999928 secs
Time difference of -9.98974e-05 secs
Time difference of -1.00136e-05 secs
Time difference of -9.53674e-07 secs
Time difference of 0 secs
Time difference of 0 secs
Time difference of 0 secs
Time difference of 0 secs
R>

Answer 2

I suspect this is a rounding/floating point issue:

Browse[2]> print(head(as.numeric(order.by)), digits = 20)
[1] 1332234170.0009999275 1332234170.0009999275 1332234170.0009999275
[4] 1332234170.0009999275 1332234170.0009999275 1332234170.0009999275

That was achieved by debugging xts() on a the call

foo <- xts(1:180, rep(as.POSIXlt("2012-03-20 09:02:50.001"), 180), 
           unqiue = FALSE)

but you can see the problem via clearly via

> print(as.numeric(as.POSIXlt("2012-03-20 09:02:50.001")))
[1] 1332234170
> print(as.numeric(as.POSIXlt("2012-03-20 09:02:50.001")), digits = 20)
[1] 1332234170.0009999275

Indicating your fractional number of seconds can't be created nor stored at exactly .001 milliseconds. Whereas truncation as 3 dp will keep .002 as it is stored as:

> print(as.numeric(as.POSIXlt("2012-03-20 09:02:50.002")), digits = 20)
[1] 1332234170.0020000935

Truncating or rounding that to 3 dp will preserve the .002 part. One of the issues you have to deal with in working with computers.

Do note that this appears to just be an issue in the printed representation of the index dates:

> print(as.numeric(index(foo)[1]), digits = 20)
[1] 1332234170.0009999275

The precision (with floating point issues) is preserved in the actual object storing the index times - you just can't see that when printing the times to the console.