1

I just started to use {fable} and the {tidyverts} family of tools and so far it's going quite well.

I'm currently interested in producing long term probabilistic forecasts from daily data (with a monthly or quarterly resolution being fine or preferred). My understanding is, that temporal aggregation could help reduce model uncertainty and also propagate known daily influences (especially holiday effects) to the e.g. quarterly level and improve accuracy that way.

For the daily data I plan on using prophet + covariates, for the higher aggregates (monthly to yearly) exponantial smoothing seems appropriate.

While I wonder if this approach seems generally promising, I am not quite sure how to structure the forecast problem using {thief} to arrive at probabilistic forecasts.

PS: I found this helpful post for hourly data, but I'm running into problems implementing it for daily data (e.g. creating meaningful aggregations and combining the forecasts): https://stats.stackexchange.com/questions/352121/how-to-forecast-hourly-as-well-as-daily-data-in-r

stats-hb
  • 958
  • 13
  • 31

1 Answers1

4

To produce probabilistic forecasts using {thief} you'll need to convert the fable into a {forecast} style forecast object. The basic minimal structure of a forecast object (and how it can be produced from a fable distribution, which can be obtained for prophet via {fable.prophet}) is:

library(forecast)
library(distributional)
dist <- dist_normal(1:10)

structure(
  list(
    # A time series of the forecast means
    mean = ts(mean(dist)),
    # A matrix time series of the forecast interval's upper bound
    upper = ts(cbind("80%" = quantile(dist, 0.90), "95%" = quantile(dist, 0.975))),
    # A matrix time series of the forecast interval's lower bound
    lower = ts(cbind("80%" = quantile(dist, 0.10), "95%" = quantile(dist, 0.025))),
    # A vector containing forecast interval levels in order of appearance
    level = c(80, 95)
  ),
  class = "forecast"
)
#>    Point Forecast      Lo 80     Hi 80       Lo 95     Hi 95
#>  1              1 -0.2815516  2.281552 -0.95996398  2.959964
#>  2              2  0.7184484  3.281552  0.04003602  3.959964
#>  3              3  1.7184484  4.281552  1.04003602  4.959964
#>  4              4  2.7184484  5.281552  2.04003602  5.959964
#>  5              5  3.7184484  6.281552  3.04003602  6.959964
#>  6              6  4.7184484  7.281552  4.04003602  7.959964
#>  7              7  5.7184484  8.281552  5.04003602  8.959964
#>  8              8  6.7184484  9.281552  6.04003602  9.959964
#>  9              9  7.7184484 10.281552  7.04003602 10.959964
#> 10             10  8.7184484 11.281552  8.04003602 11.959964

Created on 2020-12-10 by the reprex package (v0.3.0)

You can then use these forecast structures with thief::reconcilethief() to reconcile them.

That said, temporal (and cross-temporal) reconciliation is currently being developed for {fable}. The relevant development issue is here: https://github.com/tidyverts/fabletools/issues/59 There are some complexities with reconciling daily or sub-daily temporal to monthly and annual hierarchies which have not yet been implemented, but the prototype currently works well for >monthly bottom levels. This recent presentation at ISF2020 introduces temporal reconciliation with {fable}: https://www.youtube.com/watch?v=6D7rNHZ5E-Q&t=1120

It's worth noting that the intervals from {thief} are simply the sum of intervals from higher temporal frequencies. These intervals are not optimal, and so will differ from the optimal probabilistic forecast distributions that will be given by {fable}.

Mitchell O'Hara-Wild
  • 2,174
  • 5
  • 9
  • This is excellent and I am excited to try it out. Mostly out of curiosity, what are the additional complexities for weekly/daily/subdaily data? My naive first thought would be, that the allocation in the temporal hierarchy would be quite clear? (or is it about the other types of seasonality that are possible?) – stats-hb Dec 10 '20 at 17:05
  • 1
    The typical construction of summation matrices used in temporal reconciliation identifies how higher frequency intervals (1 month, 1 quarter, etc.) nest within the highest frequency interval (1 year). This is easy for monthly and quarterly aggregates, as everything nests exactly. Weeks don't exactly nest years, and so the aggregation varies by year to year. The way daily data aggregates to annual (and consequently, sub-daily) also varies due to leap years, and so the summation matrix for days into years varies by year. – Mitchell O'Hara-Wild Dec 10 '20 at 22:44
  • Dear Mr. O'Hara, I have tried to download `moment` package, but it appears it does not exist anymore. Does `aggregate_index` from `fabletools` has the same application? I also would be grateful if you could let me know where I can study about recent developments of temporal aggregation with `fable`. – Anoushiravan R Mar 19 '22 at 23:01
  • It can be found on GitHub here: https://github.com/mitchelloharawild/moment/ – Mitchell O'Hara-Wild Mar 20 '22 at 23:39