Multi processes read&write one file

Question

I have a txt file ABC.txt which will be read and wrote by multi processes. So when one process is reading from or writing to file ABC.txt, file ABC.txt must be locked so that any other processes can not reading from or writing to it. I know the enum System.IO.FileShare may be the right way to handle this problem. But I used another way which I'm not sure if it is right. The following is my solution.

I added another file Lock.txt to the folder. Before I can read from or write to file ABC.txt, I must have the capability to read from file Lock.txt. And after I have read from or written to file ABC.txt, I have to release that capability. The following is the code.

        #region Enter the lock
        FileStream lockFileStream = null;
        bool lockEntered = false;
        while (lockEntered == false)
        {
            try
            {
                lockFileStream = File.Open("Lock.txt", FileMode.Open, FileAccess.Read, FileShare.None);
                lockEntered = true;
            }
            catch (Exception)
            {
                Thread.Sleep(500);
            }
        }
        #endregion

        #region Do the work
        // Read from or write to File ABC.txt
        // Read from or write to other files
        #endregion

        #region Release the lock
        try
        {
            if (lockFileStream != null)
            {
                lockFileStream.Dispose();
            }
        }
        catch
        {
        }
        #endregion

On my computer, it seems that this solution works well, but I still can not make sure if it is appropriate..

Edit: Multi processes, not multi threads in the same process.

Accessing the same file concurrently from different threads is a really bad idea. Why not perform the IO from a single thread? You could put data to be written into a ConcurrentQueue, and dequeue the data from a dedicated writing thread. — spender, Sep 09 '13 at 02:03
why not use System.IO.FileShare since it looks like a proper solution? — Ricky, Sep 09 '13 at 02:05
@spender, thank you. ConcurrentQueue is a good idea, but as I edited my question, in fact ABC.txt is accessed by two different processes but not two threads in the same process. And I don't want to use a third process to persist the ConcurrentQueue. Do you have any idea? — Lcng, Sep 09 '13 at 06:06
@Ricky, thank you. System.IO.FileShare may be a proper solution. But if I have to write to many files, I have to write many try-catch and loop blocks in my code just as what I have done in the region "Enter the lock". I don't want to do this and it seems that this may cause starvations.. — Lcng, Sep 09 '13 at 06:13
@Boas Enkler, thank you. But "high performance" is not my worry.. — Lcng, Sep 09 '13 at 06:44
Event if not async doesn not mean new threads :-) So you don't have threading issues (no need for lock) and you don't waste threads. But its your choice — Boas Enkler, Sep 09 '13 at 06:54

bytefire · Accepted Answer · 2013-09-09T07:19:29.943

55

C#'s named EventWaitHandle is the way to go here. Create an instance of wait handle in every process which wants to use that file and give it a name which is shared by all such processes.

EventWaitHandle waitHandle = new EventWaitHandle(true, EventResetMode.AutoReset, "SHARED_BY_ALL_PROCESSES");

Then when accessing the file wait on waitHandle and when finished processing file, set it so the next process in the queue may access it.

waitHandle.WaitOne();
/* process file*/
waitHandle.Set();

When you name an event wait handle then that name is shared across all processes in the operating system. Therefore in order to avoid possibility of collisions, use a guid for name ("SHARED_BY_ALL_PROCESSES" above).

edited Sep 09 '13 at 07:19

answered Sep 09 '13 at 06:56

bytefire

4,156
2
27
36

I get this error. Access to the path '1ff907fa-6e1f-4520-bfde-d1815377c02c' is denied. – Saeed Neamati Dec 07 '15 at 07:22
Is it safe to use `using (var handle = new EventWaitHandle(...)){handle.WaitOne(); ... handle.Set();}`, or shall I wrap everything in `try{..}finally{handle.Set();}`? Will `Dispose()` set the handle free? – dmigo Jan 05 '16 at 15:18
5

Use it like this: `if(waitHandle.WaitOne())) try{ ...}finally{waitHandle.Set();}` In case of exceptions – schoetbi Nov 28 '16 at 12:08
Wouldn't it be enough with ``waitHandle.WaitOne(); try {...} finally { waitHandle.Set(); }`` (without the if part?). @schoetbi – Guimo Aug 08 '17 at 14:20
You should make note that it is important to use "EventResetMode.AutoReset" otherwise if your process crashes it could still hold the lock. – rollsch Mar 28 '21 at 00:26

score 17 · Answer 2 · answered Jun 29 '16 at 05:42

17

A mutex in C# may be shared across multiple processes. Here is an example for multiple processes writing to a single file:

using (var mutex = new Mutex(false, "Strand www.jakemdrew.com"))
{
    mutex.WaitOne();
    File.AppendAllText(outputFilePath,theFileText);
    mutex.ReleaseMutex();
}

You need to make sure that the mutex is given a unique name that will be shared across the entire system.

Additional reading here:

http://www.albahari.com/threading/part2.aspx#_Mutex

answered Jun 29 '16 at 05:42

Jake Drew

2,230
23
29

`File.AppendAllText` worked for me, but have you experimented with using `StreamWriter::WriteLine`? In my case, I don't want to open and close the file repeatedly, so I have a StreamWriter sitting around. Doing a `WriteLine` followed by a `Flush` jumbles the output – pushkin Mar 02 '20 at 18:57
Seems like you could wrap StreamWriter::WriteLine in a Mutex as well, but I have not tried it. Multiple threads / processes will always jumble the output. In my case, I was writing the output in no particular order. Each row stood alone. Perhaps you could sort the file after all processes have completed, if you have something to sort on. – Jake Drew Apr 24 '20 at 21:14
Question. What happens if you program is task manager killed in the middle of File.AppendAllText, or it has a native windows crash. You would have to restart your PC as the mutex would never be released. Fairly major problem no? – rollsch Aug 04 '22 at 05:25
The OS will automatically release the mutex for you. However, there are still a lot of bad things that could happen in this exact scenario... See: https://stackoverflow.com/questions/9168899/safe-handling-of-a-mutex – Jake Drew Aug 26 '22 at 19:25

score 4 · Answer 3 · answered Sep 09 '13 at 02:15

Your solution is error prone. You've basically implemented double-checked locking (http://en.wikipedia.org/wiki/Double-checked_locking) which can be very unsafe.

A better solution would be to either introduce thread isolation, whereby only one thread ever accesses the file and does so by reading from a queue upon which requests to read or write are placed by other threads (and of course the queue is protected by mutually exclusive access by threads) or where the threads synchronize themselves either by synchronization devices (lock sections, mutices, whatever) or by using some other file access logic (for example, System.IO.FileShare came up in a few reponses here.)

thank you. "does so by reading from a queue", maybe a queue is needed. But as I edited, I want to read from and write to file ABC.txt in two different threads. How do I persist that queue? Should I persist that queue in a third process or in ether one of the existing two process? And how to access that queue cross processes? Should I use WCF or other techs? — Lcng, Sep 09 '13 at 06:23

Jeremy Bell · Answer 4 · 2013-09-11T14:29:47.617

0

If it was me, I would install something like SQL Server Compact Edition for reading/writing this data.

However, if you want to be able to lock access to a resource that is shared between multiple processes, you need to use a Mutex or a Semaphore.

The Mutex class is a .Net wrapper around an OS Level locking mechanism.

Overview of Synchronization Primitives

edited Sep 11 '13 at 14:29

answered Sep 09 '13 at 04:13

Jeremy Bell

698
5
9

Thank you Jeremy Bell. I don't have a database in my production environment. What I am actually doing is that: I am writing a Windows Service to monitor the network usage of a specified process and write the result to file ABC.txt per 10 minutes. In that specified process, it will read it's network usage from file ABC.txt any time when it is downloading any things. In fact, file ABC.txt is accessed by two processes, not by two threads in the same process. – Lcng Sep 09 '13 at 05:52
1

The thing you are looking for is called a Mutex -http://msdn.microsoft.com/en-us/library/system.threading.mutex.aspx – Jeremy Bell Sep 09 '13 at 13:56
Thank you, @Jeremy Bell. Either [EventWaitHandle](http://msdn.microsoft.com/en-us/library/system.threading.eventwaithandle.aspx) or [Mutex](http://msdn.microsoft.com/en-us/library/system.threading.mutex(v=vs.100).aspx) can help me. Thank you for your answer. – Lcng Sep 10 '13 at 01:57

Multi processes read&write one file

4 Answers4

Linked

Related