Need function to write files incrementally

Question

I have a function to generate simulation data and write it out to csv files. This is working fine, but it does not write out the files until the function completes. I would like it to write a file at the end of each for loop. Any tips are much appreciated.

Truncated Code Block:

for (m in 1:M){

simulation code....

write.csv(userDF, file=filename, row.names=FALSE) 
}

Without a [reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) or at the least a little bit of code... add a write step at the end of each loop... — Justin, Jan 25 '13 at 19:51
... using `write.csv(..., append=TRUE)`. Look at `?write.csv` or `?write.table`. — Stephan Kolassa, Jan 25 '13 at 19:51

score 4 · Answer 1 · answered Jan 25 '13 at 20:00

4

There is a flush function. You would need to reference something of a connection type so perhaps

for (m in 1:M){
  simulation code....
  filx=file("filename")
  write.csv(userDF, file=filx, row.names=FALSE) 
  flush(filx)
}

answered Jan 25 '13 at 20:00

IRTFM

258,963
21
364
487

score 0 · Answer 2 · answered Jul 30 '19 at 20:55

I did this, to append to the same growing file on each iteration. It's definitely writing each loop!

library(readr)
startfile=TRUE  
for(index in 1:9){

  ... # make your dataframe here

  ## The first time through the loop, don't append. 
  ## Also, this will write the column names

  write_csv(as.data.frame(result),"result.csv", append=!startfile)

  ## On subsequent iterations, append your dataframe to the file
  startfile=FALSE
  }

Need function to write files incrementally

2 Answers2