4

consider the following example:

library(tidyverse)
set.seed(1)
forecast::forecast
x <- cumsum(rnorm(10))
y1 <- arima(x, order = c(1, 0, 0))
y2 <- x %>% arima(order = c(1, 0, 0))

length(fitted(y1))
[1] 10
length(fitted(y2))
[1] 0

The objects y1and y2are almost identical, the only exceptions being the slots calland series. So I guess that is where the fitted functions starts its magic.

I would really like to work with y1 instead of y2. Does anyone know an alternative function to fitted which produces the same result?

EDIT2: The above "bug" does not appear if the forecast package is not loaded into namespace (via eg. forecast::forecast). I wasnt aware that loading a package into namespace changes the behaviour of some functions.

EDIT: since the code seems not to be reproducible I add my `sessionInfo()´

R version 3.5.2 (2018-12-20)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=German_Austria.1252  LC_CTYPE=German_Austria.1252    LC_MONETARY=German_Austria.1252 LC_NUMERIC=C                    LC_TIME=German_Austria.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] forcats_0.4.0   stringr_1.3.1   dplyr_0.8.0.1   purrr_0.3.0     readr_1.3.1     tidyr_0.8.2     tibble_2.0.1    ggplot2_3.1.0   tidyverse_1.2.1 magrittr_1.5   

loaded via a namespace (and not attached):
 [1] zoo_1.8-4         tidyselect_0.2.5  urca_1.3-0        aTSA_3.1.2        haven_2.0.0       lattice_0.20-38   colorspace_1.4-0  generics_0.0.2    yaml_2.2.0        utf8_1.1.4        rlang_0.3.1       pillar_1.3.1     
[13] withr_2.1.2       glue_1.3.0        forecast_8.5      TTR_0.23-4        modelr_0.1.2      readxl_1.2.0      plyr_1.8.4        quantmod_0.4-13   timeDate_3043.102 munsell_0.5.0     gtable_0.2.0      cellranger_1.1.0 
[25] rvest_0.3.2       tseries_0.10-46   lmtest_0.9-36     parallel_3.5.2    curl_3.3          fansi_0.4.0       broom_0.5.1       xts_0.11-2        Rcpp_1.0.0        scales_1.0.0      backports_1.1.3   jsonlite_1.6     
[37] fracdiff_1.4-2    hms_0.4.2         stringi_1.3.1     grid_3.5.2        cli_1.0.1         quadprog_1.5-5    tools_3.5.2       lazyeval_0.2.1    crayon_1.3.4      pkgconfig_2.0.2   xml2_1.2.0        lubridate_1.7.4
Cettt
  • 11,460
  • 7
  • 35
  • 58
  • I can not reproduce the problem. – clemens Mar 26 '19 at 16:24
  • 3
    I'm also confused; is it possible you are using an `arima` function from a package you haven't mentioned? Because as far as I can tell, there's no `fitted` method for `stats::arima`, and so calling `fitted` on both of those objects returns `NULL`. – joran Mar 26 '19 at 16:25
  • As @joran said, both returns NULL for me as well – akrun Mar 26 '19 at 16:27
  • Sorry guys, I had the `tidyverse`package loaded but did not think it was relevant for the question. I will edit my question. – Cettt Mar 26 '19 at 16:30
  • could be that it also loads `forecast` – akrun Mar 26 '19 at 16:33
  • 1
    I have reproduced the error using stats::arima and stats::fitted – Hector Haffenden Mar 26 '19 at 16:34
  • @akrun, it seems to have something to do with the `forecast` package. Although I had not loaded it via `library` I used it like `forecast::...`. I updated the question again. Sorry for the misunderstandings. – Cettt Mar 26 '19 at 16:43
  • I would always use package::function(). Then you are sure what you use. – Christoph Mar 26 '19 at 17:12
  • @Christoph, tried that as well, same behaviour. Loading forecast into namespace changes the behaviour of `stats::fitted`. – Cettt Mar 26 '19 at 17:13

1 Answers1

6

What you've identified is a problem caused by non-standard evaluation, which is briefly mentioned in the technical note about the magrittr pipe:

The magrittr pipe operators use non-standard evaluation. They capture their inputs and examines them to figure out how to proceed. First a function is produced from all of the individual right-hand side expressions, and then the result is obtained by applying this function to the left-hand side. For most purposes, one can disregard the subtle aspects of magrittr's evaluation, but some functions may capture their calling environment, and thus using the operators will not be exactly equivalent to the "standard call" without pipe-operators.

If you look at the source for arima version of fitted you can see that you were correct in thinking that the call attribute is essential to the method's operation: 

getAnywhere(fitted.Arima)
A single object matching ‘fitted.Arima’ was found
It was found in the following places
  registered S3 method for fitted from namespace TSA
  namespace:TSA
with value

function (object, ...) 
{
    fitted = eval(object$call$x) - object$residuals
    fitted
}
<bytecode: 0x000000001e8ff4d8>
<environment: namespace:TSA>
HFBrowning
  • 2,196
  • 3
  • 23
  • 42
  • Re "some functions may capture the calling environment", I guess it is `arima` that does this, in its line `series <- deparse(substitute(x))`, while `fitted` just looks up `x` from the capture. – Frank Mar 26 '19 at 18:20
  • 1
    @Frank The call is captured using the base function `match.call()` and `fitted.Arima` uses that directly. I'm not actually sure what the purpose of saving the "series" is, but perhaps another method somewhere else uses it. – HFBrowning Mar 26 '19 at 20:08