1

I have a list of data tables that are of unequal lengths. Some of the data tables have 35 columns and others have 36.

I have this line of code, but it generates an error 

> lst <- unlist(full_data.lst, recursive = FALSE)
> model_dat <- do.call("rbind", lst)
Error in rbindlist(l, use.names, fill, idcol) : 
  Item 1362 has 35 columns, inconsistent with item 1 which has 36 columns. If instead you need to fill missing columns, use set argument 'fill' to TRUE.

Any suggestions on how I can modify that so that it works properly.

ATMA
  • 1,450
  • 4
  • 23
  • 33
  • 1
    Error says that the objects you are trying to bind don't have equal number of columns. – sm925 Mar 16 '18 at 13:34
  • As stated in the question, "Some of the data tables have 35 columns and others have 36." – ATMA Mar 16 '18 at 13:39

3 Answers3

5

Here's a minimal example of what you are trying to do.
No need to use any other package to do this. Just set fill=TRUE in rbindlist.

You can do this:

df1 <- data.table(m1 = c(1,2,3))
df2 <- data.table(m1 = c(1,2,3), m2=c(3,4,5))

df3 <- rbindlist(list(df1, df2), fill=T)

print(df3)

   m1 m2
1:  1 NA
2:  2 NA
3:  3 NA
4:  1  3
5:  2  4
6:  3  5
YOLO
  • 20,181
  • 5
  • 20
  • 40
2

Try to use rbind.fill from package plyr:

Input data, 3 dataframes with different number of columns

df1<-data.frame(a=c(1,2,3,4,5),b=c(1,2,3,4,5))
df2<-data.frame(a=c(1,2,3,4,5,6),b=c(1,2,3,4,5,6),c=c(1,2,3,4,5,6))
df3<-data.frame(a=c(1,2,3),d=c(1,2,3))

full_data.lst<-list(df1,df2,df3)

The solution

library("plyr")
rbind.fill(full_data.lst)
   a  b  c  d
1  1  1 NA NA
2  2  2 NA NA
3  3  3 NA NA
4  4  4 NA NA
5  5  5 NA NA
6  1  1  1 NA
7  2  2  2 NA
8  3  3  3 NA
9  4  4  4 NA
10 5  5  5 NA
11 6  6  6 NA
12 1 NA NA  1
13 2 NA NA  2
14 3 NA NA  3
Marcus Campbell
  • 2,746
  • 4
  • 22
  • 36
Terru_theTerror
  • 4,918
  • 2
  • 20
  • 39
2

If I understood your question correctly, I could possibly see only two options for having your data tables appended.

Option A: Drop the extra variable from one of the datasets 

table$column_Name <- NULL

Option B) Create the variable with missing values in the incomplete dataset.

full_data.lst$column_Name <- NA

And then do rbind function.

mukund
  • 553
  • 3
  • 15