Using fseek/fwrite with multiple processors

Question

I would like to use fseek/fwrite to allow multiple processors to write to different parts of a file.

This post suggests that this should work: using-fseek-fwrite-from-multiple-processes-to-write-to-different-areas-of-a-file

However, I get unpredictable results with the following code.

#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>

int main(int argc, char** argv)
{
    MPI_Init(&argc, &argv);
    
    int rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    
    FILE *file = fopen("ranks.out","wb");
    fseek(file,rank*sizeof(int),SEEK_SET);
    fwrite(&rank,sizeof(int),1,file);
    fclose(file);
    
    MPI_Finalize();
    
    return 0;
}

To run the code, test file size, and read the output, I use this Python script :

from numpy import fromfile, int32
import os
import subprocess
import sys

P = 8 if len(sys.argv) == 1 else int(sys.argv[1])
    
print(f"Running on {P:d} processors.")

arg_list = ['mpirun','-n',str(P),'stack_fseek']
output = subprocess.run(arg_list)
print(output)

stats = os.stat("ranks.out")
print(f"File size      : {stats.st_size:d} bytes")
esize = 4*P
print(f"Expected size) : {esize:d} bytes")

fout = open("ranks.out","rb")
r = fromfile(fout,dtype=int32)
fout.close()

print(r)

Running the above script several times, I get the following output :

Running on 4 processors.
CompletedProcess(args=['mpirun', '-n', '4', 'stack_fseek'], returncode=0)
File size      : 16 bytes
Expected size) : 16 bytes
[0 1 2 3]

Running on 4 processors.
CompletedProcess(args=['mpirun', '-n', '4', 'stack_fseek'], returncode=0)
File size      : 12 bytes
Expected size) : 16 bytes
[0 1 2]

Running on 4 processors.
CompletedProcess(args=['mpirun', '-n', '4', 'stack_fseek'], returncode=0)
File size      : 16 bytes
Expected size) : 16 bytes
[0 0 2 3]

Running on 4 processors.
CompletedProcess(args=['mpirun', '-n', '4', 'stack_fseek'], returncode=0)
File size      : 16 bytes
Expected size) : 16 bytes
[0 0 2 3]

Can someone explain why I am getting these unpredictable results? Should the above code work? I am testing this on MacOS.

I've tried different file modes, following this post : difference-between-r-and-w-in-fopen but get essentially the same results as above (with some modes increasing the file size with each run).

I have used MPI I/O, but was hoping for a low level solution using fseek/fwrite.

**Comments have been [moved to chat](https://chat.stackoverflow.com/rooms/252466/discussion-on-question-by-donna-using-fseek-fwrite-with-multiple-processors); please do not continue the discussion here.** Before posting a comment below this one, please review the [purposes of comments](/help/privileges/comment). Comments that do not request clarification or suggest improvements usually belong as an [answer](/help/how-to-answer), on [meta], or in [chat]. Comments continuing discussion may be removed. — Samuel Liew, Mar 12 '23 at 12:40

Donna · Answer 1 · 2023-03-08T18:42:18.113

Based on comments above (especially @AndrewHenle), a solution that produces reliable results uses open, lseek and write.

Here is a modified code. I modified the output value so that I don't write any 0s to the file intentionally.

#include <stdio.h>
#include <unistd.h>    // # lseek, write
#include <fcntl.h>     // # open

#include <mpi.h>

int main(int argc, char** argv)
{
    MPI_Init(&argc, &argv);
    
    int rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    int val = (rank+1)*111;

    int flags = O_WRONLY | O_CREAT;
    int modes = S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH;
    int file = open("ranks.out",flags, modes);
    lseek(file,rank*sizeof(int),SEEK_SET);    
    write(file,&val,sizeof(int));
    close(file);
    
    MPI_Finalize();
    
    return 0;
}

I've left off error checking, although it should of course be included in a real code.

Output indicates that no uninitialized values are present.

Running on 4 processors.
CompletedProcess(args=['mpirun', '-n', '4', 'stack_fseek_soln'], returncode=0)
File size      : 16 bytes
Expected size) : 16 bytes
[111 222 333 444]

This post describes some differences fwrite and write.

I'd welcome any comments/feedback.

It is worth noting `write(...)` (or `read(...)` fwiw) does **not** guarantee the requested number of bytes will be written, so you should "wrap" `write()` into a `while` loop to ensure all the bytes are written (even if you are unlikely to observe this if you write only four bytes at a time). — Gilles Gouaillardet, Mar 09 '23 at 00:17

Using fseek/fwrite with multiple processors

1 Answers1