1

Implemented some code from previous question: Lapply to Add Columns to Each Dataframe in a List

Using the method above, I receive corrupt data. While I cannot provide actual data, I am wondering if additional arguments need to be implemented to prevent shuffling.

Basically, this: Require: data.table

df1 <- data.frame(x = runif(3), y = runif(3))
df2 <- data.frame(x = runif(3), y = runif(3))
dfs <- list(df1, df2)
years <- list(2013, 2014)
a<-Map(cbind, dfs, year = years)
final<-rbindlist(a)

But applied to a list of thousands of data frame lists has incorrect results. Assume that some data frames, say df 1.5 somewhere between two above data frames, are empty. Would that affect the order in which the Map binds the years to the dfs? Essentially, I have an output with some data belonging to different years than the Map attached it to. I tested the length and order of years list, and compared it to the output year in final. They are identical. Any thoughts?

Community
  • 1
  • 1
sammyramz
  • 533
  • 2
  • 5
  • 13
  • 1
    Btw, it's generally good to use `set.seed` before making a random example. It makes it easier to compare and verify results. – Frank Jun 30 '16 at 14:19

2 Answers2

1

We create a logical index based on the length of each element in 'dfs', use that to subset both the 'dfs' and the 'years' and then do the cbind with Map

i1 <- sapply(dfs, length)>1

Or to make it more stringent

i1 <- sapply(dfs, function(x) is.data.frame(x) & !is.null(x) & length(x) >0 )  
a <- Map(cbind, dfs[i1], year = years[i1])

and then do the rbindlist with fill = TRUE in case the number of columns are not the same in all the data.frames in the `list.

rbindlist(a, fill = TRUE)

data

dfs[[3]] <- list(NULL)
dfs[[4]] <- data.frame()
years <- 2013:2016
akrun
  • 874,273
  • 37
  • 540
  • 662
1

Use the idcol argument to rbindlist and add the year column afterwards:

res = rbindlist(dfs, idcol=TRUE)
res[.(.id = 1:2, year = 2013:2014), on=".id", year := i.year]

X[i, on=cols, z := i.z] merges X with i on cols and then copies z from i to X.

Frank
  • 66,179
  • 8
  • 96
  • 180