3

I'm seeing some surprising behavior regarding file handles in Windows.

I have two processes both trying to acquire read-only file handles to the same underlying file.

Process A (a Go binary) opens up a read-only file handle to that file and then uses it to acquire an exclusive lock on the file.

Process B (a Python binary) then tries to open up a read-only file handle on the same file, which results in the following IOError:

IOError: [Errno 13] Permission denied: 'C:\\path\\to\\file.txt'

When Process A isn't around, Process B has no problem opening up the file.

As far as I know, there's no problem with two processes on Windows both holding read-only file descriptors to the same file, and I haven't found any documentation to suggest that one of those processes holding an exclusive lock changes that.

Does anyone know what might be causing the Python open() call to fail?

(If it's helpful, the Go file locking implementation that I'm using is this one.)

zeptonaut
  • 895
  • 3
  • 10
  • 20
  • 1
    You even say process A uses it to create an exclusive lock. – Matt_G Feb 08 '18 at 18:34
  • 1
    I think my confusion stems from the fact that, on Unix systems, opening and locking files are completely orthogonal concepts. Process B can open a file that's exclusively locked by Process A. Based on tocode's answer (which I've now accepted), this doesn't appear to be the case on Windows. – zeptonaut Feb 08 '18 at 19:54

1 Answers1

0

As suggested in this answer, open() throws an IOError either if the process doesn't have permission to access the file or if the file is locked by another process.

To verify that it's the second and not the first, you can use os.access() to check whether the process has permissions to access the file.

For example:

if not os.access(FILE_PATH, os.R_OK):
  print "Failure opening file: No read access for %s" % FILE_PATH
  return

try:                                                                                                                                                      
  fd = open(FILE_PATH)
  print "Success opening file"
except IOError:
  print "Failure opening file: Another process holds lock on %s" % FILE_PATH

With flock on Unix systems, opening and locking files are separate concepts. LOCK_FILE_EX on Windows systems, however, actually blocks other processes from opening that portion of the file for read or write access (source).

zeptonaut
  • 895
  • 3
  • 10
  • 20
tocode
  • 115
  • 1
  • 10
  • 1
    `os.access` is meaningless in this case on Windows. It only checks the file attributes, which are relevant to `F_OK` and `W_OK` (read-only attribute). As written, it has nothing to do with checking file permissions. That's best checked by calling `CreateFile` directly with the desired access. If it fails with access denied (5), you know you're not permitted the desired access. If it fails with a sharing violation (32), you know you're permitted in general; however, the file is currently open without sharing the desired read, write, or delete access. – Eryk Sun Feb 09 '18 at 01:57
  • Windows file locking in no way prevents *opening* a file. File locking is applied to regions of a file and only gets checked when attempting to read from or write to a locked region. Access to opening a file multiple times is protected by a separate I/O sharing mechanism that applies to files opened for read/execute, write/append, or delete access. See [creating and opening files](https://msdn.microsoft.com/en-us/library/aa363874) for a discussion of how the sharing mode works. – Eryk Sun Feb 09 '18 at 02:01
  • eryksun, I'm not sure your explanation explains the behavior that I'm seeing. With one golang program opening the file with read access and one python program opening the file with read access, I'd expect that they'd both be able to hold concurrent handles. However, this doesn't seem to be the case. (The golang call looks like `file, err := os.OpenFile(path, os.O_RDONLY|os.O_CREATE, 0666)`, and the Python call looks like `file = open(path)`.) – zeptonaut Feb 09 '18 at 15:35
  • @candrews, I can tell you that standard Python 2 `open` calls Microsoft C `fopen` / `_wfopen`, which ultimately calls `CreateFile` with the share mode set to allow read and write sharing. Offhand, I don't know how golang implements `OpenFile`. If it doesn't set the share mode to allow read sharing, then `open` in Python will fail with a permission error. It's actually a sharing violation (winerror 32) instead of access denied (winerror 5), but Microsoft C makes life difficult for us by mapping thousands of Windows error codes down to about 50 POSIX error codes, such as `EACCES` (13). – Eryk Sun Feb 09 '18 at 16:12
  • 1
    @candrews, I took a look at the locking library you're using. Its Windows implementation is not based on actual file locking (i.e. `LockFileEx`, `UnlockFileEx`) to lock regions of a file. Instead it calls `CreateFile` using the share mode as a 'lock' if `share` is false, and otherwise allows read and write sharing (but not delete). The only 'unlock' possible in this case is to close the file handle. – Eryk Sun Feb 09 '18 at 16:38