0

I am running R simulation using a multi-core system. The simulation result I am monitoring is a vector of length 900. My plan was to append this vector (row-wise) in text file (using write.table) once each simulation ends. My simulation run from 1:1000. While I was working on my laptop the result was fine because the work is sequential. When I work i cluster the simulations are divided and there might be a conflict on who to write first. The reason for my claim is I am even getting even impossible values for the first column of my text file (This column used to store the simulation index). If you need a sample code I can attach.

lmo
  • 37,904
  • 9
  • 56
  • 69
belaya
  • 15
  • 4
  • 1
    Here is a [related post](http://stackoverflow.com/questions/22104858/is-it-a-good-idea-to-read-write-files-in-parallel) with the `foreach` package. Maybe the [comments here](http://stackoverflow.com/questions/20425071/lock-file-when-writing-to-it-from-parallel-processes-in-r) are related. Also [this post](http://stackoverflow.com/questions/27909677/parallel-simulations-writing-on-the-same-file), though the second is for `julia`. – lmo Feb 06 '17 at 20:31
  • Thank you. The comment given in your second suggestion are helpful – belaya Feb 06 '17 at 20:45

1 Answers1

1

There is no way to write to a text file with parallel threads that respects order. Each thread has it's own buffer and has no indication when it is appropriate to write, because there is no cross communication. So what will happen is they'll all try to the same file at the same time, even in the middle of another thread's write which is why you're getting impossible values in your first column.

The solution is to write to separate files for each thread, or return the value as the output of the multithreaded apply loop. Then combine the results at the end sequentially.

thc
  • 9,527
  • 1
  • 24
  • 39