Trouble using object in dataframe after a pipe (decomposition of a msts object)

Question

I do time series decomposition and I want to save the resulting objects in a dataframe. It works if I store the results in a object and use it to make the dataframe afterwards:

# needed packages
library(tidyverse)
library(forecast)
# some "time series"
vec <- 1:1000 + rnorm(1000)
# store pipe results
pipe_out <-
# do decomposition
  decompose(msts(vec, start= c(2001, 1, 1), seasonal.periods= c(7, 365.25))) %>%
# relevant data
  .$seasonal
# make a dataframe with the stored seasonal data
data.frame(ts= pipe_out)

But doing the same as a one-liner fails:

decompose(msts(vec, start= c(2001, 1, 1), seasonal.periods= c(7, 365.25))) %>%
  data.frame(ts= .$seasonal)

I get the error

Error in as.data.frame.default(x[[i]], optional = TRUE, stringsAsFactors = stringsAsFactors) : cannot coerce class ‘"decomposed.ts"’ to a data.frame

I thought that the pipe simply moves forward the things that came up in the last step which saves us storing those things in objects. If so, shouldn't both codes result in the very same output?

EDIT (from comments)

The first code works but it is a bad solution because if one wants to extract all the vectors of the decomposed time series one would need to do it in multiple steps. Something like the following would be better:

decompose(msts(vec, start= c(2001, 1, 1), 
seasonal.periods= c(7, 365.25))) %>%
      data.frame(seasonal= .$seasonal, x=.$x, trend=.$trend, random=.$random)

Answer 1

It's unclear from your example whether you want to extract $x or $seasonal. Either way, you can extract part of a list either with the `[[`() function in base or the alias extract2() in magrittr, as you prefer. You should then use the . when you create a data.frame in the last step.

Cleaning up the code a bit to be consistent with the piping, the following works:

library(magrittr)
library(tidyverse)
library(forecast)

vec <- 1:1000 + rnorm(1000)

vec %>%
  msts(start = c(2001, 1, 1), seasonal.periods= c(7, 365.25)) %>%
  decompose %>%
  `[[`("seasonal") %>%
  # extract2("seasonal") %>% # Another option, uncomment if preferred
  data.frame(ts = .) %>%
  head # Just for the reprex, remove as required
#>            ts
#> 1 -1.17332998
#> 2  0.07393265
#> 3  0.37631946
#> 4  0.30640395
#> 5  1.04279779
#> 6  0.20470768

^{Created on 2019-11-28 by the reprex package (v0.3.0)}

Edit based on comment:

To do what you mention in the comments, you need to use curly brackets (see e.g. here for an explanation why). Hence, the following works:

library(magrittr)
library(tidyverse)
library(forecast)

vec <- 1:1000 + rnorm(1000)

vec %>% 
  msts(start= c(2001, 1, 1), seasonal.periods = c(7, 365.25)) %>%
  decompose %>%
  {data.frame(seasonal = .$seasonal,
              trend = .$trend)} %>%
  head
#>     seasonal trend
#> 1 -0.4332034    NA
#> 2 -0.6185832    NA
#> 3 -0.5899566    NA
#> 4  0.7640938    NA
#> 5 -0.4374417    NA
#> 6 -0.8739449    NA

However, for your specific use case, it may be clearer and easier to use magrittr::extract and then simply bind_cols:

vec %>% 
  msts(start= c(2001, 1, 1), seasonal.periods = c(7, 365.25)) %>%
  decompose %>%
  magrittr::extract(c("seasonal", "trend")) %>%
  bind_cols %>%
  head
#> # A tibble: 6 x 2
#>   seasonal trend
#>      <dbl> <dbl>
#> 1   -0.433    NA
#> 2   -0.619    NA
#> 3   -0.590    NA
#> 4    0.764    NA
#> 5   -0.437    NA
#> 6   -0.874    NA

^{Created on 2019-11-29 by the reprex package (v0.3.0)}

Answer 2

With daily data, decompose() does not work well because it will only handle the annual seasonality and will give relatively poor estimates of it. If the data involve human behaviour, it will probably have both weekly and annual seasonal patterns.

Also, msts objects are not great for daily data either because they don't store the dates explicitly.

I suggest you use tsibble objects with an STL decomposition instead. Here is an example using your data.

library(tidyverse)
library(tsibble)
library(feasts)

mydata <- tsibble(
  day = as.Date(seq(as.Date("2001-01-01"), length=1000, by=1)),
  vec = 1:1000 + rnorm(1000)
)
#> Using `day` as index variable.
mydata
#> # A tsibble: 1,000 x 2 [1D]
#>    day           vec
#>    <date>      <dbl>
#>  1 2001-01-01  0.161
#>  2 2001-01-02  2.61 
#>  3 2001-01-03  1.37 
#>  4 2001-01-04  3.15 
#>  5 2001-01-05  4.43 
#>  6 2001-01-06  7.35 
#>  7 2001-01-07  7.10 
#>  8 2001-01-08 10.0  
#>  9 2001-01-09  9.16 
#> 10 2001-01-10 10.2  
#> # … with 990 more rows

# Compute a decomposition
mydata %>% STL(vec) 
#> # A dable:           1,000 x 7 [1D]
#> # STL Decomposition: vec = trend + season_year + season_week + remainder
#>    day           vec trend season_year season_week remainder season_adjust
#>    <date>      <dbl> <dbl>       <dbl>       <dbl>     <dbl>         <dbl>
#>  1 2001-01-01  0.161  14.7       -14.6      0.295    -0.193           14.5
#>  2 2001-01-02  2.61   15.6       -14.2      0.0865    1.04            16.7
#>  3 2001-01-03  1.37   16.6       -15.5      0.0365    0.240           16.9
#>  4 2001-01-04  3.15   17.6       -13.0     -0.0680   -1.34            16.3
#>  5 2001-01-05  4.43   18.6       -13.4     -0.0361   -0.700           17.9
#>  6 2001-01-06  7.35   19.5       -12.4     -0.122     0.358           19.9
#>  7 2001-01-07  7.10   20.5       -13.4     -0.181     0.170           20.7
#>  8 2001-01-08 10.0    21.4       -12.7      0.282     1.10            22.5
#>  9 2001-01-09  9.16   22.2       -13.8      0.0773    0.642           22.9
#> 10 2001-01-10 10.2    22.9       -12.7      0.0323   -0.0492          22.9
#> # … with 990 more rows

^{Created on 2019-11-30 by the reprex package (v0.3.0)}

The output is a dable (decomposition table) which behaves like a dataframe most of the time. So you can extract the trend column, or either of the seasonal component columns in the usual way.