open with O_CREAT - was it opened or created?

Question

I have 10 processes which try open the same file more or less at the same time using open(O_CREAT) call, then delete it. Is there any robust way to find out which process actually did create the file and which did open already create file, for instance, if I want to accurately count how many times that file was opened in such scenario.

I guess I could put a global mutex on file open operation, and do a sequence of open() calls using O_CREAT and O_EXCL flags, but that doesn't fit my definition of "robust".

Use `(O_CREAT|O_EXCL)` to get an error if the file already exists. When you get the error, you check the `errno` to see if it is because it does exist, then re-open however you want to open it, knowing that it already exists. — jxh, Apr 03 '13 at 21:29
And then do what ? But what if another process opens it after my check but before my "re-open however I want" ? — Sergey, Apr 03 '13 at 21:30
Your problem in your description is not fully specified then. Update your question with the actual problem you are facing. Show some code, and point out where something is not happening the way you expect. — jxh, Apr 03 '13 at 21:37
The classic idiom, before there was an O_CREAT, was to call `open()` to open an existing file and `creat()` to create it if the `open()` failed. The [`creat()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/creat.html) function is supposed to be implemented as if it was `int creat(const char *path, mode_t mode) { return open(path, O_WRONLY|O_CREAT|O_TRUNC, mode); }` so I don't particularly recommend it, but there is no other way to know whether you created a new file or opened an existing one (while doing either). There's a TOCTOU issue with the open/creat or open/open technique. — Jonathan Leffler, Apr 03 '13 at 21:38
@user315052: except that you don't *know* that it already exists, because by the time you come around again it might have been deleted/renamed. — Steve Jessop, Apr 03 '13 at 22:00
@SteveJessop: That is why the problem description is incomplete. — jxh, Apr 03 '13 at 22:00
@user315052: I think the problem description is *reasonably* complete. It's asking for an add-or-update transaction that tells you which it did. The issue is that Posix doesn't provide that as an atomic operation. So if the questioner wants us to design his software for him then we need more information, but if he'll take "no" for an answer then the question is asked, understood, and answered :-) — Steve Jessop, Apr 03 '13 at 22:03
@Sergey: If you are open to it, can you describe the use case for this operating behavior? Perhaps I can talk you into a solution that doesn't require deleting the file. — jxh, Apr 03 '13 at 22:52
@user315052 I was just dealing with some windows code which was actually checking if it has created the file or opened an existing one, and thought linux should have something similar. — Sergey, Apr 03 '13 at 23:06
If that's the case, you can ignore the extra information that windows provides and just use the `O_CREAT` behavior on linux to get same semantics on both systems. Cross-platform development usually ends up falling back on the least common denominator. — jxh, Apr 03 '13 at 23:13
To @user315052 _Cross-platform development usually ends up falling back on the least common denominator_ or creating a code like posted below to emulate missing features, which is what I favor doing to keep things simple on the other side. — Sergey, Apr 03 '13 at 23:19

score 9 · Answer 1 · edited Apr 03 '13 at 23:16

9

Use O_EXCL flag with O_CREAT. This will fail if the file exists and errno will be set to EEXIST. If it does fail then attempt open again without O_CREAT and without O_EXCL modes.

e.g.

int fd = open(path, O_WRONLY | O_CREAT | O_EXCL, 0644);
if ((fd == -1) && (EEXIST == errno))
{
    /* open the existing file with write flag */
    fd = open(path, O_WRONLY);
}

edited Apr 03 '13 at 23:16

Jonathan Leffler

730,956
141
904
1,278

answered Apr 03 '13 at 21:30

suspectus

16,548
8
49
57

can you please clarify "attempt to open again without O_CREAT". If I omit O_CREAT from your code, I'll be left with O_EXCL only, and accorting to [this](http://linux.die.net/man/2/open) "In general, the behavior of O_EXCL is undefined if it is used without O_CREAT". – Sergey Apr 03 '13 at 21:36
If the O_EXCL create fails because EEXISTS, try again without the _`O_EXCL`_ and without the _`O_CREAT`_! Note that when you have O_CREAT, `open()` takes 3 arguments; the third is the file permissions mode (0644 or similar). You also need one of O_RDONLY, O_WRONLY or O_RDWR (though omitting them is equivalent to O_RDONLY, but creating a file in read-only mode is modestly pointless). – Jonathan Leffler Apr 03 '13 at 21:43
I've amended the post which hopefully will clarify. – suspectus Apr 03 '13 at 21:44
ok, but while the processor is executing the comments right after the if-check, and someone else opens my file, what do I do then ? – Sergey Apr 03 '13 at 21:45
@Sergey: Then your problem description is not complete, so we can't answer it. – jxh Apr 03 '13 at 21:46
@Sergey? What's your goal? Multiple processes can have the same file open. If you use O_APPEND, you can get atomic writes at the end. Why do you care whether it existed or not that isn't handled by the options to [`open()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html)? – Jonathan Leffler Apr 03 '13 at 21:47
Ok, please help me create a complete description. I've marked the important part with bold, what else is missing ? – Sergey Apr 03 '13 at 21:48
@Sergey: We don't know what problem you actually encountered. We don't know what problem you are trying to solve. We don't know what your code is trying to do. Only you can tell us that. – jxh Apr 03 '13 at 21:49
@Sergey you will need to check that the second `open()` is successful and handle open failure appropriately. – suspectus Apr 03 '13 at 21:50
@JonathanLeffler, I have 10 processes trying to open the file, which might no exist yes. I want to know, if it is the case, which one process did create the new file, and which other 9 did open existing file. – Sergey Apr 03 '13 at 21:50
@Sergey: My suggestion is that you attempt to solve that problem with the answer suspectus provided. The process that succeeds with the first open knows who it is. The other nine will fail on the first open. You make code to do the rest. If there is an issue you cannot resolve on your own, bring it back in a separate question, showing the code, describe what it was supposed to do, and what it does instead. – jxh Apr 03 '13 at 21:56
solution posted by @suspectus kinda works, but my concern is that it is not 100% safe. File can be deleted in-between 2 lines and then 2nd open will fail. I guess I need to mention that in original question. To make this solution be bulletproof, a retry loop needs to be introduced to guard the code against deletions. – Sergey Apr 03 '13 at 22:07

jxh · Accepted Answer · 2013-04-03T23:30:39.520

Based roughly on your comments, you want something along the lines of this function:

/* return the fd or negative on error (check errno);
   how is 1 if created, or 0 if opened */
int create_or_open (const char *path, int create_flags, int open_flags,
                    int *how) {
    int fd;
    create_flags |= (O_CREAT|O_EXCL);
    open_flags &= ~(O_CREAT|O_EXCL);
    for (;;) {
        *how = 1;
        fd = open(path, create_flags);
        if (fd >= 0) break;
        if (errno != EEXIST) break;
        *how = 0;
        fd = open(path, open_flags);
        if (fd >= 0) break;
        if (errno != ENOENT) break;
    }
    return fd;
}

This solution is not bullet proof. There may be cases (symbolic links maybe?) that would cause it to loop forever. Also, it may live-lock in certain concurrency scenarios. I'll leave resolving such issues as an exercise. :-)

In your edited question, you pose:

I have 10 processes which try open the same file more or less at the same time using open(O_CREAT) call, then delete it.

A hack-ish, but more bullet proof, solution would be to give each process a different user ID. Then, just use the regular open(path, O_CREAT|...) call. You can then query the file with fstat() on the file descriptor, and check the st_uid field of the stat structure. If the field equals the processes' user ID, then it was the creator. Otherwise, it was an opener. This works since each process deletes the file after opening.

To fight live-lock I'd fallback to some sort of global shared mutex after a few iterations. But then the solution becomes something I was hoping to avoid. — Sergey, Apr 03 '13 at 22:36
I accept this answer since this is the closest to what I wanted to achieve. — Sergey, Apr 04 '13 at 10:49
In this case, but also in general, consider compiling your programs with [`_FORTIFY_SOURCE`](https://stackoverflow.com/a/16604146/11509478) set to 1 which adds [some checks](https://stackoverflow.com/a/56201991/11509478) at compile-time only. — , May 20 '19 at 10:29

open with O_CREAT - was it opened or created?

2 Answers2