0

I'm looking for an async-friendly, cross-process read/write lock for .NET. I've found various solutions that are 'async friendly' (like AsyncReaderWriterLock in Stephen Cleary's excellent AsyncEx library) and examples (like this on SO) that are cross-process friendly, via Semaphores or a Mutex, but not both.

The problem to be solved is synchronizing write-access to a single file across multiple processes:

  • Multiple/unblocked readers, single writer.
  • 1+ reads block writes
  • 1+ writes block reads & writes.

Currently, my solution involves using a Semaphore to lock writes, and another to track reads, such that the first read locks the write Semaphore, and the last unlocks it.

To do this and be async-friendly I'm using WaitHandleAsyncFactory from the AsyncEx library. Though this solution seems to work, I'd prefer to use something more battle-tested, if it exists.

Does anyone know of something in the wild?

EDIT: The problem being solved is there are a set of files that are expensive to create (actually the output of a compiler) that need to be accessed by two different applications on the same machine. A user can initiate the compilation from either at the same time, and even in parallel. The approach that was taken was to have a zip file with cache keyed folders that contain these results. Safely checking/populating the contents of this zip, from these multiple applications, is the problem at hand.

pianomanjh
  • 233
  • 1
  • 14
  • 2
    So, if I gave you some code, how would you determine it was battle tested? I am assuming anything I created is not, and anything on SO is not, so you are looking for a library? Unfortunately your question criteria seems a little subjective – TheGeneral Sep 27 '21 at 23:40
  • The closest thing you're going to get to "battle tested", is .NET BCL named system wait handles, ie `EventWaitHandle`, `Semaphore` etc. But os handles are not inherently async so you want to use a wrapper around a named handle like `Semaphore(string)`. I recommend taking another look at your question and narrow down what you're looking for, as it stands your question is very vague and subjective. – DekuDesu Sep 28 '21 at 00:12
  • What are you trying to do? Most likely you don't need any kind of lock. A `Channel` would be far better. On the other hand, `cross-process` depends on OS features like named mutexes that are handled by the OS itself, not .NET. Mutex is cross-platform while `Semaphore` and `EventWaitHandle` are Windows-only – Panagiotis Kanavos Sep 28 '21 at 08:03
  • In the [cross-process memory mapped file example](https://learn.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files#non-persisted-memory-mapped-files) a named Mutex is used to synchronize access between processes. This is the only cross-platform option. – Panagiotis Kanavos Sep 28 '21 at 08:10
  • What are you trying to do? Perhaps instead of trying to modify (and lock) the same resource you could use a queue, message passing, a database. Perhaps even a SQLite database in [WAL Mode](https://sqlite.org/wal.html) – Panagiotis Kanavos Sep 28 '21 at 11:03
  • Do you intend to create a single async cross-process lock for all the involved processes, or multiple locks? In the first case, you could consider creating an asynchronous wrapper around a `Semaphore` or `Mutex`, so that only one thread of each process will be blocked waiting for the lock (when there is contention). – Theodor Zoulias Sep 28 '21 at 11:17
  • @PanagiotisKanavos The problem to be solved is synchronizing write-access to a single file across multiple processes. Multiple/unblocked readers, single writer. 1+ reads block writes, 1+ writes block reads & writes. As linked and referenced in the question, I'd like what the AsyncReaderWriterLock achieves, but across local machine processes. – pianomanjh Sep 28 '21 at 20:58
  • @pianomanjh I posted a link that shows how to synchronize multiple processes. The only cross-platform option is Mutex. Semaphore and EventWaitHandle are Windows-only. And none of them is async. You could emulate `async` by starting *another* thread to block and signal a TaskCompletionSource, but that would just move the block to another thread. You'd lose the benefit of async – Panagiotis Kanavos Sep 29 '21 at 07:15
  • @pianomanjh what is the *actual* problem though? Your description is too low-level. What file? What does it do? How is it used? SQLite for example allows concurrent writers and readers through its WAL (write-ahead-logging) mode. Instead of everyone writing to the same file, changes are recored to the log and applied to the data asynchronously. Readers read the *data*, so neither is blocked (up to a point). The problem you describe is more complex than just synchronizing reads and writes – Panagiotis Kanavos Sep 29 '21 at 07:25
  • @pianomanjh what does single writer mean? Only one process writes, eg to a log? Or can multiple processes write, but only one at a time? In the first case, instead of locking, the writer could emit a signal to the readers to pick up the new data. Or it could emit that data along with the notification. Or perhaps emit the file offset of the data change. There are low-level OS-specific ways to do this, or high-level cross-platform ways like WebSockets/gRPC/SignalR notifications. – Panagiotis Kanavos Sep 29 '21 at 07:29
  • @pianomanjh if you want to read the tail of a log file it could be enough [for readers to open the file in FileShare.ReadWrite mode](https://stackoverflow.com/a/3791142/134204). – Panagiotis Kanavos Sep 29 '21 at 07:33
  • @pianomanjh please explain the actual problem *in the question itself*. The question has attracted 2 downvotes and will soon close. I don't think you're asking for recommendations and a solution to the *actual* problem is possible. Just not in the way you think. Operating systems have no async locking mechanisms. There *are* higher-level mechanisms though – Panagiotis Kanavos Sep 29 '21 at 08:02
  • @pianomanjh `I'm using WaitHandleAsyncFactory from the AsyncEx library` this is probably better than anything anyone else could propose already, and you probably should post that code in the question itself, if not as an actual answer. The `ThreadPool.RegisterWaitForSingleObject` call inside that method is better than anyone else could propose, eg using `Task.Run` to call `EnterRead`, or trying to indirectly use a background thread with `ConfigureAwait(false)` or `Task.Yield` – Panagiotis Kanavos Sep 29 '21 at 08:39
  • @PanagiotisKanavos I've updated the question with the higher level detail about the 'why'. The question posed here is not an ask for alternative caching solutions (we are considering obvious alternatives), but an ask if anyone out there is aware of reliable .NET code to solve the cross process file access problem. If the answer is "We don't know of any" then I can pursue alternatives. – pianomanjh Sep 30 '21 at 16:18
  • @pianomanjh I suspect you solved this in the past 18 months. For the actual problem mentioned I'd build a single service that would queue requests and produces the files instead of trying to synchronize 2-3 different applications. It's relatively easy to build eg a .NET 6 minimal API that receives and queues requests, uses a queued job to do the actual work (maybe just an ActionBlock). Checking for existence can be done through polling at the very least. If you create a gRPC service you'll be able to send notifications back to the callers when the files are done – Panagiotis Kanavos Jan 23 '23 at 15:22

1 Answers1

1

The following CrossProcessLock class should be close to what you are searching for. It's a combination of a cross-process Semaphore and an in-process SemaphoreSlim. The result is a non-thread-affine synchronization primitive, with a potentially blocking EnterAsync method:

public class CrossProcessLock : IDisposable
{
    private readonly Semaphore _globalSemaphore;
    private readonly SemaphoreSlim _localSemaphore;

    public CrossProcessLock(string name)
    {
        _globalSemaphore = new Semaphore(1, 1, name);
        _localSemaphore = new SemaphoreSlim(1, 1);
    }

    public async Task EnterAsync()
    {
        await _localSemaphore.WaitAsync().ConfigureAwait(false);
        try { _globalSemaphore.WaitOne(); }
        catch { _localSemaphore.Release(); throw; }
    }

    public void Exit()
    {
        _globalSemaphore.Release();
        _localSemaphore.Release();
    }

    public void Dispose()
    {
        _globalSemaphore.Dispose();
        _localSemaphore.Dispose();
    }
}

Usage example:

var gate = new CrossProcessLock("MyLock");
await gate.EnterAsync();
try
{
    await RingTheBellsAsync();
}
finally { gate.Exit(); }

The EnterAsync method may block in case the internal _globalSemaphore is currently acquired by another process, and no other in-process asynchronous flow is currently waiting for the lock. If you want to ensure that the current thread will not be blocked, you'll have to offload the invocation to the ThreadPool with the Task.Run method:

await Task.Run(() => gate.EnterAsync());

Only one ThreadPool thread may be blocked at maximum, per CrossProcessLock instance.


Update: And here is a cross-process async reader-writer lock, based on ideas originated from this question: Cross-process read-write synchronization primitive in .NET?

public class CrossProcessAsyncReaderWriterLock : IDisposable
{
    private readonly Semaphore _globalReader;
    private readonly Semaphore _globalWriter;
    private readonly SemaphoreSlim _localReader;
    private readonly SemaphoreSlim _localWriter;
    private readonly int _maxConcurrentReaders;

    public CrossProcessAsyncReaderWriterLock(string name, int maxConcurrentReaders)
    {
        _globalReader = new Semaphore(
            maxConcurrentReaders, maxConcurrentReaders, name + ".Reader");
        _globalWriter = new Semaphore(1, 1, name + ".Writer");
        _localReader = new SemaphoreSlim(1, 1);
        _localWriter = new SemaphoreSlim(1, 1);
        _maxConcurrentReaders = maxConcurrentReaders;
    }

    public async Task EnterReaderAsync()
    {
        await _localReader.WaitAsync().ConfigureAwait(false);
        try
        {
            _globalWriter.WaitOne();
            _globalReader.WaitOne();
            _globalWriter.Release();
        }
        finally { _localReader.Release(); }
    }

    public void ExitReader()
    {
        _globalReader.Release();
    }

    public async Task EnterWriterAsync()
    {
        await _localWriter.WaitAsync().ConfigureAwait(false);
        try
        {
            _globalWriter.WaitOne();
            for (int i = 0; i < _maxConcurrentReaders; i++) _globalReader.WaitOne();
            _globalWriter.Release();
        }
        finally { _localWriter.Release(); }
    }

    public void ExitWriter()
    {
        _globalReader.Release(_maxConcurrentReaders);
    }

    public void Dispose()
    {
        _globalReader.Dispose();
        _globalWriter.Dispose();
        _localReader.Dispose();
        _localWriter.Dispose();
    }
}

This one uses two local SemaphoreSlims, and so it can potentially block two threads at maximum, per CrossProcessAsyncReaderWriterLock instance.

The maxConcurrentReaders specifies how many concurrent cross-process readers are allowed. Important: all processes should be configured with the same number. Setting this value too high may result to degraded performance. Setting it too low may result to increased contention.

Usage example:

var gate = new CrossProcessAsyncReaderWriterLock("MyRWLock", 10);
await gate.EnterReaderAsync();
try
{
    await ReadSomethingAsync();
}
finally { gate.ExitReader(); }

Again, if keeping the current thread non-blocked is necessary, the Task.Run could be used:

await Task.Run(() => gate.EnterReaderAsync());

I've stress-tested this class in a multithreaded console application, with multiple instances of the app running concurrently, and it seems that its read-write invariants are well enforced in the realm of each individual process. I haven't tested it with actual system resources though.

Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
  • Thanks, do you have a version that does not block multiple readers? https://github.com/StephenCleary/AsyncEx/blob/master/doc/AsyncReaderWriterLock.md describes what I'm trying to achieve, but cross-process – pianomanjh Sep 28 '21 at 21:01
  • @pianomanjh I added a reader-writer implementation. – Theodor Zoulias Sep 29 '21 at 00:14
  • 1
    This code still blocks a thread, even though it's not the original thread. The whole point of using asynchronous operations is to *avoid* blocking threads. Otherwise, one could use a TaskCompletionSource. Besides, `Semaphore` is only available on Windows – Panagiotis Kanavos Sep 29 '21 at 07:17
  • @PanagiotisKanavos feel free to post a cross-platform solution to this problem, that doesn't block any threads. – Theodor Zoulias Sep 29 '21 at 07:34
  • @TheodorZoulias what problem? We still don't know what the actual problem is, only a low-level description of the possible solution. There's nothing that's both cross-process *and* async-friendly out of the box – Panagiotis Kanavos Sep 29 '21 at 07:37
  • 1
    @PanagiotisKanavos this answer is an attempt to implement this "low-level description of the possible solution". Feel free to interrogate the OP until they spew the actual problem that you think that they are hiding from us, but not here please. You'd better do the interrogation under their question, not under my answer. – Theodor Zoulias Sep 29 '21 at 08:02
  • @TheodorZoulias which others and I have been doing since yesterday. The OP asked for an async solution. This isn't one, it just wraps the blocking solution with an async-y API. Which may be just fine, but is *not* async – Panagiotis Kanavos Sep 29 '21 at 08:06