0

I write a function to reshape a logfile(600Mb) to get a dataframe, but I failed to get expectable result. The problem is I can get the expectable result when I run a logfile of about 10Mb, but it failed when I run the full file, it seem was led by memory allocation.

How do I fix it? 

Reshape_data <-function(X) {
  n<-length(X[,1])
  Date<-vector(length=n)
  Time_Temp<-vector(length=n)
  status_Temp<-vector(length=n)
  m <- 0
  i <- 1
  while(is.na(X[i,]) == FALSE) { 
    if (substr(X[i,],1,1)!="#") {            
      m<-m+1
      Date[m]<-sapply((strsplit(as.character(X[i,])," ")), "[", 1) 
      Time_Temp[m]<-sapply((strsplit(as.character(X[i,])," ")), "[", 2) 
      status_Temp[m]<-sapply((strsplit(as.character(X[i,])," ")), "[", 11)
    }        
    i <- i+1
  }
  if(m>0){
    Time_Temp<-Time_Temp[1:m]
    status_Temp<-status_Temp[1:m]
  }else 
    {
      Time_Temp<-NULL 
      status_Temp<-NULL
    }
  mydf<-data.frame(Time_Temp,status_Temp)
  return(mydf)
}

AA_log<-read.table("ex130828.log",sep="\t",stringsAsFactors=FALSE,encoding="utf-8")
AA_Tidy<-Reshape_data(AA_log)
PascalVKooten
  • 20,643
  • 17
  • 103
  • 160
David Wang
  • 17
  • 4
  • Without input data we can't help you. Please create a reproducible example. See the [FAQ](http://stackoverflow.com/a/5963610/1412059) for advice how to do that. However, I can tell you that R is not C and you shouldn't use a `while` loop for things like this. – Roland Oct 09 '13 at 13:19
  • Simillar to [**this problem**](http://stackoverflow.com/a/16136390/1478381)? – Simon O'Hanlon Oct 09 '13 at 13:38
  • Welcome to SO. Your code is difficult to read, so I'm not going to attempt to fathom what you are trying to do, but if you want to reshape data then `melt` and `dcast` in the `reshape2` package are a good place to start. – Richie Cotton Oct 09 '13 at 14:43

0 Answers0