c++ multiprocess writing a unique file

Question

I have a process which creates multiple fork processes and each forked process creates a file with the same name. But the requirement is to create only one file by any of the forked processes.

Can you please suggest how to make sure file is created by only one process.

I tried to check if the file exists before writing a file but it is not working as multiple processes are getting pass beyond that check.

Check the return value from `fork` to see which process is the parent and which is the child, then only have one of them do the file creation depending on that value. — Blaze, Oct 29 '18 at 10:04
There can be multiple child , each child creating same file. — EmptyData, Oct 29 '18 at 10:06
what are you going to do with the file after creation? do you need to write to it, or just need it to exist? — slawekwin, Oct 29 '18 at 10:23
@slawekwin after each child process exit then I will read from the file. — EmptyData, Oct 29 '18 at 10:24
That's also what parallel file systems are for. Especially writing to the same file. — Matthieu Brucher, Oct 29 '18 at 10:39
XY problem most likely. Why doesn't it matter *which* child writes the file? Please explain what you're implementing exactly. — rustyx, Oct 29 '18 at 11:09

Maxim Egorushkin · Accepted Answer · 2018-10-29T10:29:54.260

2

Can you please suggest how to make sure file is created by only one process.

Each of the concurrent processes (or threads) creates its own file in the target directory with a unique temporary filename with mkstemp.
Then the process renames that file to the target filename. Renaming is atomic and either succeeds or fails. If it fails, then another process already created the file. In this case the temporary file can be removed and the file with the target filename should be opened instead.

edited Oct 29 '18 at 10:29

answered Oct 29 '18 at 10:24

Maxim Egorushkin

131,725
17
180
271

I solved it using `fopen` with `wx` mode similar to `open` with `O_EXCL`. – EmptyData Oct 29 '18 at 11:15
@EmptyData With `fopen(..., "wx")` you do not have a chance to initialize the file before renaming it to its target name. – Maxim Egorushkin Oct 29 '18 at 11:19

Aconcagua · Answer 2 · 2018-10-29T11:01:07.013

You might open the file in the parent process before forking. Each child then can, according to this answer, just re-use the file descriptor and you should be fine.

Of course, you'd still need some means to synchronise your different processes such that writes to the file are interleaved properly (one process writing AB and the other one C resulting in ABC or CAB, but not ACB...). Doing so is easier with threads, so if you can use them you might prefer them. Otherwise, this answer might be of interest to you.

Edit (according to new comments to question): If I understand correctly, you want to do the following:

Parent process creates several child processes.
Each child writes some information to a file.
After each or all children having terminated, parent reads back the information from files.

You might do the following then:

Parent process opens a new temporary file with with arbitrary name for each child - right before forking. This way, according to above, parent and child have the same file descriptors available.
After forking, child process uses the file descriptor to write it's information to, parent process pushes the descriptor to e. g. a std::vector or, if more suitable, a std::map mapping PID to FD.
Parent process waits for children, on termination uses the file descriptors stored to read information back.

Have you considered using pipes instead? They might be the better alternative...

c++ multiprocess writing a unique file

2 Answers2