13

Problem:

After a close() syscall that fails with EINTR or EIO it is unspecified whether the file has been closed. (http://pubs.opengroup.org/onlinepubs/9699919799/) In multi-threaded applications, retrying the close may close unrelated files opened by other threads. Not retrying the close may result in unusable open file descriptors piling up. A clean solution might involve invoking fstat() on the freshly closed file descriptor and a quite complex locking mechanism. Also, serializing all open/close/accept/... invocations with a single mutex may be an option.

These solutions do not take into account that library functions may open and close files on their own in an uncontrollable way, e.g., some implementations of std::thread::hardware_concurrency() open files in the /proc filesystem.

File Streams as in the [file.streams] C++ standard section are not an option.

Is there a simple and reliable mechanism to close files in the presence of multiple threads?

edits:

Regular Files: While most of the time there will be no unusable open file descriptors accumulating, two conditions might trigger the problem: 1. Signals emitted at high frequency by some malware 2. Network file systems that lose connection before caches are flushed.

Sockets: According to Stevens/Fenner/Rudoff, if the socket option SO_LINGER is set on a file descriptor referring to a connected socket, and during a close(), the timer elapses before the FIN-ACK shutdown sequence completes, close() fails as part of the common procedure. Linux does not show this behavior, however, FreeBSD does, and also sets errno to EAGAIN. As I understand it, in this case, it is unspecified whether the file descriptor is invalidated. C++ code to test the behavior: http://www.longhaulmail.de/misc/close.txt The test code output there looks like a race condition in FreeBSD to me, if it's not, why not?

One might consider bocking signals during calls to close().

Heiko Bloch
  • 388
  • 2
  • 15
  • 1
    Related: http://stackoverflow.com/q/33114152/694576 – alk Oct 26 '16 at 09:14
  • I believe that with EINTR the state is well-defined. You just have to repeat the call again. – Victor Dyachenko Oct 26 '16 at 12:23
  • 2
    @VictorDyachenko POSIX says the state of fd is unspecified on both EINTR and EIO. – P.P Oct 26 '16 at 13:12
  • 3
    On Linux, close unconditionally invalidates its argument filedescriptor even when it "fails". Linus mentioned it on the mailing list once. – Petr Skocik Oct 26 '16 at 14:50
  • 3
    Here's the archive of the mail: http://yarchive.net/comp/linux/close_return_value.html – Petr Skocik Oct 26 '16 at 14:57
  • Regarding your examples (edits): 1) If there's a malware that has the privilege to send signals to your process then it'll most likely do something "better" than repeatedly sending interruptible signals (EINTR) ;-) 2) In NFS, even a successful `close()` doesn't guarantee that data is written to disk either i.e. even after a successful close(), cached data on the client might still be lost if connection is gone. So, these examples are quite theoretical IMO. Besides, POSIX provides *no* alternative/solution either. – P.P Oct 26 '16 at 15:25
  • 2
    See also [Not checking `close()`'s return value — how serious really?](http://stackoverflow.com/questions/19056309/), [How to close a file?](http://stackoverflow.com/questions/22603025/), [System call interrupted by a signal still has to be completed](http://stackoverflow.com/questions/13356497) and no doubt others (I couldn't find the one I was expecting to find). Basically, you're stuck. You have to treat the file descriptor as if it is closed — never use it again unless it is returned by a file descriptor creating function (in which case, you can safely assume that the close was OK). – Jonathan Leffler Oct 27 '16 at 15:56

3 Answers3

8

This issue has been fixed in POSIX for the next issue; unfortunately it's too big a change to have made it into the recent TC2. See the final accepted text for Austin Group Issue #529.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
3

There's no practical solution for this problem as POSIX doesn't address this at all.

Not retrying the close may result in unusable open file descriptors piling up.

As much as it sounds like legitimate concern, I have never seen this happen due to failed close() calls.

A clean solution might involve invoking fstat() on the freshly closed file descriptor and a quite complex locking mechanism.

Not really. When close() failed, the state of the file descriptor is unspecified. So, you can't reliably use it a fstat() call. Because the file descriptor might have been closed already. In that case, you are passing an invalid file descriptor to fstat(). Or another thread might have reused it. In that case, you are passing the wrong file descriptor to fstat(). Or the file descriptor might have been corrupted by the failed close() call.

When process exits, all the open descriptors will be flushed and closed anyway. So, this isn't much of a practical concern. One could argue that this would be a problem in a long running process in which close() fails too often. But I have seen this happen in my experience and POSIX doesn't provide any alternative either.

Basically, you can't do much about this except report that the problem occurred.

P.P
  • 117,907
  • 20
  • 175
  • 238
  • The only potential solution that I can think of would be to call `fstat()` *before* calling `close()`, and if the `close()` call failed, call `fstat()` again and try to determine if it's the same file/object at the other end of the descriptor. But that only works if you know that there's no way another thread could have opened the same file/object. And as you've already noted, this isn't a huge problem - I've never seen a `close()` call fail. – Andrew Henle Oct 26 '16 at 10:18
2

To mitigate any issues, explicitly sync the file:

  1. (If you are operating on FILE*, first call fflush() on it to make sure user space buffers are emptied to kernel.)
  2. Call fsync() on the file descriptor, to flush any kernel data and metadata about the file to disk.

These you can retry on error without extra worries. After that, possibly leaking file descriptors or handles on interrupted close on some OSes is probably a minor issue, especially if you check the behavior for OSes which are important to you (I suspect there's no problem in most relevant OSes).

Also, once the file and data are flushed, the chances of getting interrupted during close is much smaller, as then close should not actually touch disk. If you do get EIO or EINTR anyway, just (optionally) log it and ignore it, because doing anything else probably does more harm than good. It's not a perfect world.

hyde
  • 60,639
  • 21
  • 115
  • 176