1

I have a big buffer with integer data (a big integer array) and I want to write it into a file.

However I am using openMP and my entire application is now multithreaded. So I wonder, after the buffer is full, is there a way for me to use openMP and have multiple threads writting into the same file making several read-only access to the buffer?

If with openMP it is impossible, is it possible to do it any other way? How should I do it?

Are there any libraries you know for doing this?

talonmies
  • 70,661
  • 34
  • 192
  • 269
Flame_Phoenix
  • 16,489
  • 37
  • 131
  • 266
  • 2
    Assuming you aren't doing anything fancy formatting-wise when you write the ints, this is going to be a very IO bound operation, so multi-threading isn't going to help. Ie, you are limited by how fast your disk can write, which won't improve with more threads telling it what to write. I recommend using `#pragma omp single` to limit writing to the file by single thread. You will have to take care that other threads don't modify your array while you are writing it. – Nicu Stiurca Mar 17 '13 at 17:22
  • Thanks! I will perform I/O with only 1 thread then. – Flame_Phoenix Mar 17 '13 at 18:43

1 Answers1

3

Multiple threads writing to a file will make the process even more slow than it already is, since this will force many seek operations (which are the slowest ones, considering default magnetic HDD's).

Consider writing the data to the disk with just one thread, but the reading being made with multiples threads accessing directly the in memory array. Once that array is not allocated anymore, the access should be performed at the disk, causing again a big slow down due to the seek times. To make reduce the slow down, you must implement some buffering scheme or use only one thread.

There are other things to be considered depending on how are your readings performed and your algorithm implemented, but I believe these are some of the general lines.

Vinícius Gobbo A. de Oliveira
  • 2,239
  • 23
  • 26