0

I want to read a csv file of 4000 columns and 3000 rows and rows are of different length. Now i'm using the code below to read, but the maximum number of columns can be read is 2067.

read_data <- function(filename) {
  setwd(dir)
  no_col <- max(count.fields(filename, sep = ","))
  temp_data <- read.csv(filename, header = FALSE, sep = ",", row.names = NULL, na.strings = 0, fill = TRUE, col.names=1:no_col)

How do I solve this problem?

Community
  • 1
  • 1
Caroline Wang
  • 69
  • 3
  • 5
  • 1
    You might want to try `fread` from the `data.table` package or `read.csv.ffdf` from the `ff` package. – Jaap Oct 18 '15 at 09:19
  • 1
    btw: could you include the error message you get? – Jaap Oct 18 '15 at 09:20
  • 1
    Not quite a duplicate, but see https://stackoverflow.com/questions/1727772/quickly-reading-very-large-tables-as-dataframes-in-r – Richie Cotton Oct 18 '15 at 09:26
  • Even read.csv should just work. Try reading in the file without your function. – phiver Oct 18 '15 at 09:28
  • Manual of read.table (function used by read.csv) says: ‘read.table’ is not the right tool for reading large matrices, especially those with many columns: it is designed to read _data frames_ which may have columns of very different classes. Use ‘scan’ instead for matrices. – Kamil S Jaron Oct 18 '15 at 10:05
  • why are you assigning numeric column names? R will create incrementing column names for you if none exist in the data file. – hrbrmstr Oct 18 '15 at 12:04

0 Answers0