.net file access synchronisation

Question

I'm trying to synchronise a webapi that's going to serve images. When I request a specific image, it will check whether the image already exists and return it if it does, or if it doesn't exist it will create it and then return it.

My problem is that it's obviously not thread safe just like that; I have one thread come in and determine the image doesn't exist and starting to create it, while another request comes in and also determines the image doesn't exist (just yet) and tries to create it as well. I know I could lock the whole thing to avoid the issue, but I'm trying to avoid that. There will be 100,000s of images, and I don't see why I need to stop all threads from reading the other images just because one image doesn't exist yet. Is there a "usual" way of doing this? Images are requested by ids, can I lock on the id of a particular image? For example

List<long> _locks = new List<long>();
_locks.Add(17);
lock(_locks[0]){...}

It just doesn't look right... surely there's a better solution?

What you can do is, on your `Read` add 3 re-try cycles, 2 seconds apart. So, if you receive `File access exception`, presumably when file is written, just wait and try reading again. — T.S., Feb 13 '14 at 07:39
@T.S. thanks mate, I had thought of that and it's a "backup" solution I guess, but I'd like to try and avoid the exception in the first place if I can. I haven't done much in terms of multi threading so just trying to figure out how to do it properly I guess. Tried locking the whole section on one (static) object, and the avg response time goes from 20ms to 200ms...!! — Andy, Feb 13 '14 at 08:05

score 2 · Accepted Answer · edited May 23 '17 at 10:24

2

It depends on your server setup (single or farm). On a single server you can use a shared HashSet. The filename is an excellent key. You only need a short lock around the Set, that won't hurt performance.

Here is a question about a concurrent HashSet

edited May 23 '17 at 10:24

Community

1
1

answered Feb 13 '14 at 11:24

H H

263,252
30
330
514

This worked well in the end. It brought the avg response down to 12ms - certainly better than 200ms with lock() and better than no management (and therefore plenty of errors) and 20ms. Thanks for your help. – Andy Feb 19 '14 at 00:14

score 0 · Answer 2 · edited May 23 '17 at 12:18

My friend, you want to avoid "try-based" approach, you want to avoid "locking" approach and yet, you accessing a shared resource. Try to call corflags on any managed dll and leave the command window up. Then go to windows explorer and try to delete that file. You will get "the file is in use". Even Windows is not totally out of the woods with files, with all its caching mechanisms. Of course, the problem is that you have all files go through same lock. May be you can do something about that will minimize that condition. Lets see. This is totally untested idea:

Create custom lock object

public class CustomLock
{
    private object _lock = new object();
    private string _file;

    public CustomLock(string file)
    {_file = file;}

    public object Read()
    {
        lock(_lock)
        {
            // read your file
        }
    }

    public object Read()
    {
        lock(_lock)
        {
            // read your file
        }
    }

    public override int GetHashCode()
    {
        // here your hash algorithm
    }

}

Now, use HashTable with HashTable.Synchronized as described here http://msdn.microsoft.com/en-us/library/system.collections.hashtable.synchronized%28v=vs.110%29.aspx, to have a cache of your locks

Or you can synch the Dictionary What's the best way of implementing a thread-safe Dictionary?. So, now it is going to be something like this:

 var custLock = GetCustLock(file); // This should always return CustomLock object, new or existing
 object file = custLock.Read();

Presumably, using this design, you will remove bottleneck of locking all files in same place. Please Test it.

.net file access synchronisation

2 Answers2