0

everyone.

What I'm trying to do

To create an empty ff data.frame in R.

Details

I'd like to read multiple csv files in R, bind them together and create one big data.frame. Since the data are very huge, I'm using ff package.

Here is my code.

file_list = list.files(pattern="*.csv")
library(ff)
for(i in seq_along(length(file_list)){
  ffdf <- read.csv.ffdf(x=ffdf, file=file_list[i], header=T)
}  

However, I got the following error.

Error in `rownames<-`(x, value) : 
  attempt to set 'rownames' on an object with no dimensions

I searched the error message in Google and Stackoverflow but got no useful result. Does anyone know how to deal with this issue?

Updated (15/2/16)

The following code worked.

library(ffbase)
library(ff)
file_list = list.files(pattern="*.csv")
lst <- lapply(file_list, function(x) read.csv.ffdf(file=x,header=TRUE))
ff1 <- Reduce(function(x,y) ffdfappend(x,y, adjustvmode=F), lst)

At first, the lappy row was like this, which didn't work.

lst <- lapply(file_list, read.csv.ffdf, header=TRUE)

The key was writing file=. It seems like ff function requires indicating attributes explicitly. (Reference : Import text file using ff package)

Thanks to all of you!

My environment

  • Windows 7 Home Premium Service Pack 1
  • R studio 0.98.1091
  • R version 3.1.2 (2014-10-31)
Community
  • 1
  • 1
dixhom
  • 2,419
  • 4
  • 20
  • 36
  • "attempt to set 'rownames' on an object with no dimensions" indicates you have nothing in your csv file. In that case, don't add the data (ff is about large data, not about no data) –  Feb 10 '15 at 15:06

1 Answers1

0

If the intention is to create a big ffdf object from multiple files,

library(ffbase)
library(ff)
ff1 <- Reduce(function(x,y) ffdfappend(x,y, adjustvmode=F), lst)
dim(ff1) #from the example
#[1] 78  7

where

lst <- lapply(file_list, read.csv.ffdf, header=TRUE)

data

Using the example dataset from the ?read.csv.ffdf

 x <- data.frame(log=rep(c(FALSE, TRUE), length.out=26),
   int=1:26, dbl=1:26 + 0.1
 , fac=factor(letters), ord=ordered(LETTERS)
 , dct=Sys.time()+1:26, dat=seq(as.Date("1910/1/1"), length.out=26, by=1))
 x <- x[c(13:1, 13:1),]
 csvfile <- tempPathFile(path=getOption("fftempdir"), extension="csv")
 write.csv(x, file=csvfile, row.names=FALSE)
 y <- read.csv(file=csvfile, header=TRUE)
 y
 cat("Read csv with header\n")
 ffx <- read.csv.ffdf(file=csvfile, header=TRUE)
 lst <- lapply(1:3, function(x) read.csv.ffdf(file=csvfile, 
       header=TRUE))
 dim(lst[[1]])
 #[1] 26  7
akrun
  • 874,273
  • 37
  • 540
  • 662
  • Thanks, akrun. I tried the code. But I got an error (original post updated). I'll try to work it out. – dixhom Feb 08 '15 at 13:39
  • @dixhom Could you try to read on a single dataset and see if the error persists.? i.e. `read.csv.ffdf(filelist[1], header=TRUE)` – akrun Feb 08 '15 at 13:45
  • The code didn't work. However, I managed to solve the original problem. Main post updated. – dixhom Feb 16 '15 at 14:41