Long format into list, listFromLong results in: argument 1 is not a vector

Question

I have been trying to run the MSCMT::listFromLong command to transform a data frame from long format to list. I am sure that my data is in long format, however whenever I run the code I run into the following error

Error in order(rownames(res[[i]])) : argument 1 is not a vector

After looking at the listFromLong's source code I realized that the error must be somewhere along the lines between ** ** in this code:

> listFromLong <- function(foo, unit.variable, time.variable, 
>                          unit.names.variable=NULL,exclude.columns=NULL) {  
> if(!is.data.frame(foo)) stop("foo must be a data.frame")
> 
>   # main helper function   DFtoList <-
> function(input,rowcol,colcol,colnamecol=NULL,exclude=NULL) {
>     stopifnot(length(dim(input))==2)
>     datcols    <- setdiff(seq_len(ncol(input)), 
>                           c(rowcol,colcol,colnamecol,exclude))
>     *res        <- vector("list",length(datcols))*
>     names(res) <- if (!is.null(colnames(input))) colnames(input)[datcols] else 
>                                                  as.character(datcols)
>     if (!is.null(colnamecol)) {
>       c2n        <- na.omit(unique(input[,colnamecol]))
>       names(c2n) <- na.omit(unique(input[,colcol]))
>     }  
>     for (i in seq_along(res)) {
>       idx  <- !is.na(input[,datcols[i]])
>       rown <- unique(input[idx,rowcol])
>       coln <- unique(input[idx,colcol])
>       res[[i]] <- matrix(NA,nrow=length(rown),ncol=length(coln))
>       rownames(res[[i]]) <- rown
>       colnames(res[[i]]) <- coln
>       for (j in which(idx)) 
>         res[[i]][as.character(input[j,rowcol]),as.character(input[j,colcol])]
> <- 
>           input[j,datcols[i]]
>       **if (!is.null(colnamecol)) colnames(res[[i]]) <- c2n[as.character(coln)]**
> **      res[[i]] <- res[[i]][order(rownames(res[[i]])),,drop=FALSE]**
>     }
>     res   }

I can't pinpoint the exact source of the error as, to my understanding, the data is ready to be processed by this code. I would appreciate it if someone could tell me how to locate "Argument 1", or give me an explanation as to why it was not converted to a vector in the line between ** **.

My data was imported from Stata using

data <- readstata13::read.dta13("path/data.dta") 

Answer 1

I'm not sure where the error occurs, but as far as I see the function basically splits the data along the index "regionname" into matrices using "year" as row names. After that, it transposes things so that there is a list of matrices of the original columns with now countries in the columns.

We could do the same very simple using an array, transposing it accordingly using aperm, and finally convert into a list using split.along.dim() from @flodel's answer. Example:

data(basque, package="Synth")
B <- split.along.dim(aperm(simplify2array(by(basque, basque$regionname, function(x) {
  `rownames<-`(as.matrix(x[,!names(x) %in% c("regionno", "regionname", "year")]),
               x[,"year"])})), c(1, 3, 2)), 3)

With your original data you would just replace the data and column names.

Now let's check if the result is equal to that of MSCMT::listFromLong.

Basque <- MSCMT::listFromLong(basque, unit.variable="regionno", 
                              time.variable="year", 
                              unit.names.variable="regionname")

Weirdly, MSCMT::listFromLong deletes all rows of the list elements with NAs, but let's do it...

B <- lapply(B, function(x) x[apply(x, 1, function(x) !all(is.na(x))), ])

Columns of our B are sorted alphabetically, we do that with Basque.

Basque <- lapply(Basque, function(x) x[, order(colnames(x))])

Now check,

all.equal(Basque, B)
# [1] TRUE

et voilà!

---

Function of @flodel's answer, that transforms an array into a list:

split.along.dim <- function(a, n) {
  setNames(lapply(split(a, arrayInd(seq_along(a), dim(a))[, n]),
                  array, dim = dim(a)[-n], dimnames(a)[-n]),
           dimnames(a)[[n]])
}