Is mkdir still atomic? (Windows 7 filesystems mounted on SAN)

Question

We have some old applications that communicate via directory-based queues. Each item in the queue is a file, and there's a header file that maintains an ordered list of the filenames for the items in the queue.

Naturally, this old code needs to lock the queue while items are pushed and popped. What it's doing is creating a lock subdirectory, on the assumption that mkdir() is an atomic operation - if multiple processes attempt to create a directory, only one of them is going to succeed.

One of my co-workers has been trying to chase down an obscure problem, and he thinks the causes is that this locking is no longer working, when the processes are running on different machines, and when the filesystem in question is mounted on a SAN.

Is there any possibility that he might be correct?

I've experienced some odd behaviour (since moving to Windows 7/2008R2) related to creating and/or removing directories on a file server. I haven't been able to nail the details down, but it wouldn't surprise me if your co-workers observation was correct. — Harry Johnston, Jun 11 '12 at 23:54
Suggestion: change the code so that when it creates a directory in this way, it also adds a file with a randomly generated (but identifiable) name. If it later sees two such files in a single directory, you've confirmed that the locking is not working. — Harry Johnston, Jun 11 '12 at 23:57
I'm not sure that would work. Seems like the same sort of low-level optimizations that are allowing two mkdirs to happen would be just as likely to allow one program to not see that the other has created a file, until it's too late. — Jeff Dege, Jun 12 '12 at 14:54
I don't mean this as a solution, just as a possible way of confirming that what you think might be happening is in fact happening. While it wouldn't prove that it isn't happening, it might prove that it is! Another possibility would be to create a global log file on the same share (with appropriate explicit locking semantics) recording directory creation and removal. — Harry Johnston, Jun 12 '12 at 22:12

score 1 · Answer 1 · answered Nov 27 '17 at 00:52

Very old question I know, but I hope someone finds this interesting.

I was also getting confusing results using PowerShell to create a shared folder for use as a mutex, so I created a test script.

function New-FolderMutex {
    try {
        New-Item -ItemType directory -Path .\TheMutex -ErrorAction Stop > $null
        $true
    } catch {
        $false
    }
}

function Remove-FolderMutex {
    Remove-Item -Path .\TheMutex
}

1..100 | % {
    if (New-FolderMutex) {
        Write-Host "Inside loop $_"
        Remove-FolderMutex
    }
}

This script is run while the current directory is in a network share.

When I ran this script simultaneously in two separate PowerShell consoles, it was clear from the error messages that the approach was doomed. There are a number of different errors produced by the call to Remove-Item, even though it is being called only by the process that created the folder. It seems that behind the scenes there are a whole bunch of non-atomic steps occurring.

Of course, the OP was asking about mkdir (probably the system call) and my example uses the much higher level PowerShell cmdlets, but I hope this is of some interest.

Example output from one of two processes (edited for brevity)

Inside loop 30
Inside loop 31
Inside loop 32
Inside loop 33
Inside loop 34
Remove-Item : Access to the path 'H:\My Documents\PowerShell\MutexFolder\TheMutex' is denied.
At H:\My Documents\PowerShell\MutexFolder\Test-FolderMutex.ps1:93 char:5
+     Remove-Item -Path .\TheMutex
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : PermissionDenied: (H:\My Documents...Folder\TheMutex:String) [Remove-Item], UnauthorizedAccessException
    + FullyQualifiedErrorId : RemoveItemUnauthorizedAccessError,Microsoft.PowerShell.Commands.RemoveItemCommand

Inside loop 39
Remove-Item : H:\My Documents\PowerShell\MutexFolder\TheMutex is a NTFS junction point. Use the Force parameter to delete or modify.
At H:\My Documents\PowerShell\MutexFolder\Test-FolderMutex.ps1:93 char:5
+     Remove-Item -Path .\TheMutex
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (H:\My Documents...Folder\TheMutex:DirectoryInfo) [Remove-Item], IOException
    + FullyQualifiedErrorId : DirectoryNotEmpty,Microsoft.PowerShell.Commands.RemoveItemCommand

Inside loop 42
Remove-Item : Could not find a part of the path 'H:\My Documents\PowerShell\MutexFolder\TheMutex'.
At H:\My Documents\PowerShell\MutexFolder\Test-FolderMutex.ps1:93 char:5
+     Remove-Item -Path .\TheMutex
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (H:\My Documents...Folder\TheMutex:String) [Remove-Item], DirectoryNotFoundException
    + FullyQualifiedErrorId : RemoveItemIOError,Microsoft.PowerShell.Commands.RemoveItemCommand

Inside loop 44
Inside loop 45

Is mkdir still atomic? (Windows 7 filesystems mounted on SAN)

1 Answers1

Linked