I have a 5GB .csv file to be analysed, I was thinking of reading it chunk by chunk nad than process a chunk at a time. so was trying to skip those many lines which are already been read. this code is not working, it only reads first chunk successfully, after that it doesn't reads garbage value for the following iterations. need help with the code...
ReadCsvChunk <- function(ffName,chunk.num,chunk.size){
cnt = 1
nskip = 0
rCnt = chunk.size
repeat {
x <<- read.csv(file = ffName, nrows = rCnt, skip = nskip, header = T)
# ...
# analyse here
#....
nskip = nskip + chunk.size
if(cnt == chunk.num | nrow(x) < chunk.size){
break
}
cnt = cnt+1
} #repeat
}#function
ReadCsvChunk("WHO.csv",2,10) # read 2 chuck each of 10 rows
