3

Right now, i'm trying to make a generic IsoStorageManager, which would read/write and serialize/deserialize classes async, based on this analytic. However, i'm aware of the case when different threads would request same file for reading/writing.

My thought on this field:

  1. Just wrap all Reads()/Writes() into a lock. - Not too good, as i dont need to wait for writing different files.

  2. Add somekind of concurrent queue for writers. If same file is already processing, writer should decide if it wants to cancel previous task (rewriting) or get previous task from cache and add own changes (merge). If reader would want to access the same file, just return data from a queue. - Seems to be overcomplicated.

  3. Force readers to use one thread for all writers. Then, i'd have no problems with multiplie attempts of access to the same file. - Seems good as a temporary solution, that's the main question here. Actually, it is the same as 1.

  4. EDIT1: Maybe i need a thread-safe dictionary? Once file is going to be written, i'd store its name and data in a dictionary, so readers would just get the data from the writer itself.

Any suggestions?

EDIT2:

I'm using a task

public static async Task<T> ReadJsonAsyncTask<T>(this JsonTextReader reader)
    {
        return await TaskEx.Run(() => jsonSerializer.Deserialize<T>(reader));
    }

like this

public static async Task<T> ReadJsonEx<T>(String fileName)
    {
        if (String.IsNullOrEmpty(fileName))
            return default(T);

        return await await Task.Factory.StartNew(async () =>
        {
            using (var store = IsolatedStorageFile.GetUserStoreForApplication())
            using (var stream = new IsolatedStorageFileStream(fileName, FileMode.Open, store))
            using (var sr = new StreamReader(stream))
            using (var jr = new JsonTextReader(sr))
                return await jr.ReadJsonAsyncTask<T>();
        });
    }

The same for writer, and i want to be sure that no file would be accessed during the process.

EDIT3: aha, looking like i found an answer here: Easy way to save game in WP7 Silverlight?

EDIT4: that would work only with sync calls. Not my case. :(

EDIT5: after a whole day of search, i had found AsyncReaderWriterLock. Usage is trivial:

private static readonly AsyncReaderWriterLock readerLocker = new AsyncReaderWriterLock(); 

public static async Task<T> ReadJsonEx<T>(String fileName)
    {
        if (String.IsNullOrEmpty(fileName))
            return default(T);

        return await await Task.Factory.StartNew(async () =>
        {
            using (var locker = await readLocker.ReaderLockAsync())
            using (var store = IsolatedStorageFile.GetUserStoreForApplication())
            using (var stream = new IsolatedStorageFileStream(fileName, FileMode.Open, store))
            using (var sr = new StreamReader(stream))
            using (var jr = new JsonTextReader(sr))
                return await jr.ReadJsonAsyncTask<T>();
        });
    }

Sometimes it works, sometimes - not.

EDIT6: Ok, here are some more details about my AsyncReaderWriterLock testing case. I have reader like mentioned before and a writer which uses its own AsyncReaderWriterLock. I have a page with a progressbar and the button. Button command is:

SimpleLogger.WriteLine("Starting generation...");

        var list = new List<Order>();
        //for (var i = 0; i < 10; i++)
            list.Add(GenerateOrder());


        SimpleLogger.WriteLine("Writing 5 times the same file...");

        var res1 = await IsoStorageManager.WriteJsonEx(fileName1, list);
        var res2 = await IsoStorageManager.WriteJsonEx(fileName1, list);
        var res3 = await IsoStorageManager.WriteJsonEx(fileName1, list);
        var res4 = await IsoStorageManager.WriteJsonEx(fileName1, list);
        var res5 = await IsoStorageManager.WriteJsonEx(fileName1, list);

        SimpleLogger.WriteLine("Writing 5 different files");

        var res11 = await IsoStorageManager.WriteJsonEx(fileName1, list);
        var res12 = await IsoStorageManager.WriteJsonEx(fileName2, list);
        var res13 = await IsoStorageManager.WriteJsonEx(fileName3, list);
        var res14 = await IsoStorageManager.WriteJsonEx(fileName4, list);
        var res15 = await IsoStorageManager.WriteJsonEx(fileName5, list);

        SimpleLogger.WriteLine("Reading 5 times the same");

        var res21 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName1);
        var res22 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName1);
        var res23 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName1);
        var res24 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName1);
        var res25 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName1);

        SimpleLogger.WriteLine("Reading 5 times different");

        var res31 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName1);
        var res32 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName2);
        var res33 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName3);
        var res34 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName4);
        var res35 = await IsoStorageManager.ReadJsonEx<List<Order>>(fileName5);

        SimpleLogger.WriteLine("Done");

If to press button once, its more or less ok (different files are not writing simultaneously as it should be in a prefect world, but let it be so for now):

09:03:38.262 [00:00:00.000] Starting generation...
09:03:38.300 [00:00:00.025] Writing 5 times the same file...
09:03:43.126 [00:00:04.811] Writing 5 different files
09:03:47.303 [00:00:04.163] Reading 5 times the same
09:03:50.194 [00:00:02.871] Reading 5 times different
09:03:53.341 [00:00:03.130] Done

If to press the button several times to emulate highload and mixing of writing/reading stuff, i got this output:

08:51:52.680 [00:00:00.000] Starting generation...
08:51:52.722 [00:00:00.028] Writing 5 times the same file...
08:51:52.795 [00:00:00.057] Starting generation...
08:51:52.854 [00:00:00.043] Writing 5 times the same file...
08:51:52.892 [00:00:00.023] Starting generation...
08:51:52.922 [00:00:00.016] Writing 5 times the same file...
08:51:52.943 [00:00:00.006] Starting generation...
08:51:52.973 [00:00:00.016] Writing 5 times the same file...
08:52:06.009 [00:00:13.022] Writing 5 different files
08:52:06.966 [00:00:00.942] Writing 5 different files
08:52:07.811 [00:00:00.778] Writing 5 different files
08:52:08.513 [00:00:00.689] Writing 5 different files
08:52:22.115 [00:00:13.567] Reading 5 times the same
08:52:22.887 [00:00:00.755] Reading 5 times the same
08:52:23.773 [00:00:00.754] Reading 5 times the same

And an exception in line using (var stream = new IsolatedStorageFileStream(fileName, FileMode.Open, store))

System.IO.IOException occurred
_HResult=-2147024864
_message=[IO.IO_SharingViolation_File] 
Arguments: Folder//TestFile1.txt
Debugging resource strings are unavailable. Often the key and arguments provide sufficient information to diagnose the problem. See http://go.microsoft.com/fwlink/?linkid=106663&Version=4.0.50829.0&File=mscorlib.dll&Key=IO.IO_SharingViolation_File
HResult=-2147024864
Message=[IO.IO_SharingViolation_File]
Arguments: Folder//TestFile1.txt
Debugging resource strings are unavailable. Often the key and arguments provide sufficient information to diagnose the problem. See http://go.microsoft.com/fwlink/?linkid=106663&Version=4.0.50829.0&File=mscorlib.dll&Key=IO.IO_SharingViolation_File
Source=mscorlib
StackTrace:
    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
InnerException: 

EDIT7: tried awaitable critical section. Provides similar to AsyncReaderWriterLock result for a single command (button tap):

03:12:05.213 [00:00:00.000] Starting generation...
03:12:05.252 [00:00:00.023] Writing 5 times the same file...
03:12:09.894 [00:00:04.626] Writing 5 different files
03:12:13.700 [00:00:03.792] Reading 5 times the same
03:12:16.831 [00:00:03.115] Reading 5 times different
03:12:20.032 [00:00:03.171] Done

But also this seems to be more stable with a crash-test (4 fast button taps): it managed to finish tasks without a crash.

EDIT8: moved all that shit to the Google Spreadshit. Should have a nice public access (no login is needed). Will move all statistics there in a while.

Community
  • 1
  • 1
Vitalii Vasylenko
  • 4,776
  • 5
  • 40
  • 64
  • 1) You can also just do nothing, and let your users synchronize if they need to. 2) In what situations does `AsyncReaderWriterLock` not work? – Stephen Cleary May 11 '14 at 01:28
  • 1) What do you mean? 2) Just updated the answer. If you want, i can commit a sample to the codeplex. – Vitalii Vasylenko May 11 '14 at 09:08
  • @StephenCleary maybe i should make some kind of ConcurrentQueue and store all Tasks there? And in case a Task with a new name (filename) is added, i can fire it instantly (and keep the filename till Task is done) and in case Task with same name already exists in a queue (file is already in process), i'll add a new Task to the queue? Hm, actually, it should be not a Queue, but just a List. – Vitalii Vasylenko May 11 '14 at 09:20
  • @StephenCleary ah crap, now i'm returning to your idea about ConcurrentDictionary :) – Vitalii Vasylenko May 11 '14 at 09:29
  • Looks to me like you're missing `FileShare.Read`, which informs the OS that you want to allow other simultaneous readers. I'm not 100% familiar with the WinRT file APIs, but I would suspect that `FileShare.None` is the default, preventing simultaneous readers. – Stephen Cleary May 12 '14 at 12:41
  • And what I meant by "do nothing" is to just document that any synchronization is the responsibility of your users. If someone's just using your library for simple read-on-startup and write-on-shutdown, then they won't need synchronization at all. But if someone *needs* simultaneous reads/writes to the same file, then let them handle that complexity. – Stephen Cleary May 12 '14 at 12:43
  • @StephenCleary Actually, i'm doing this lib for myself. I need sometimes to read/write some data that i got from the server. And i just want to be sure that app would never crash, for example if user would pull the same data twice (so 2 writers would try to access same file). – Vitalii Vasylenko May 12 '14 at 13:23

1 Answers1

2

As I've understood you are planning to make a Task which you will be able to run asynchronously. IMO the best would be here Mutex - it is designed to guard shared resources from multiple access:

When two or more threads need to access a shared resource at the same time, the system needs a synchronization mechanism to ensure that only one thread at a time uses the resource. Mutex is a synchronization primitive that grants exclusive access to the shared resource to only one thread. If a thread acquires a mutex, the second thread that wants to acquire that mutex is suspended until the first thread releases the mutex.

The other advantage of Mutex is that in can be Global - you can use it to guard access to a file between processes.

As far as I remember in WP8.0 writing to a file, serialization is done synchronously, then as it is run on one thread - there will be no problem in obtaining and releasing Mutex.

Here you can find a good pattern.

EDIT

I still don't get what you try to achieve and where the problem lies. IMO you are redirecting your Deserialization to ThreadPool Thread, then you can make the code there synchronous (it's not running on UI thread) and use Mutex. Probably there are many other solutions but maybe this will help:

public static async Task<T> ReadJsonEx<T>(String fileName)
{
    if (String.IsNullOrEmpty(fileName)) return default(T);

    string mutexName = "dependantOnApp" + fileName;
    return await Task.Run<T>(() =>
    {
        using (Mutex myMutex = new Mutex(false, mutexName))
        {
            try
            {
                myMutex.WaitOne();
                using (var store = IsolatedStorageFile.GetUserStoreForApplication())
                using (var stream = new IsolatedStorageFileStream(fileName, FileMode.Open, store))
                using (var sr = new StreamReader(stream))
                using (var jr = new JsonTextReader(sr))
                    return jsonSerializer.Deserialize<T>(jr);
            }
            catch { throw new Exception("Exception"); }
            finally { myMutex.ReleaseMutex(); }
        }
    });
}
Community
  • 1
  • 1
Romasz
  • 29,662
  • 13
  • 79
  • 154
  • Thanks for idea and for a link, that should help – Vitalii Vasylenko May 10 '14 at 16:06
  • //As far as I remember in WP8.0 writing to a file, serialization is done synchronously, then as it is run on one thread - there will be no problem in obtaining and releasing Mutex. - that's what i want to achieve: to have a stable async reader/writer. – Vitalii Vasylenko May 10 '14 at 16:15
  • @VitaliiVasylenko What methods are you planning to use to write and serialize to a file? Are they async? You can make a Task that will for example use TaskCompletitionSource, it can be run asynchronously but the code inside goes synchronously. – Romasz May 10 '14 at 16:17
  • I had used this thread ( http://stackoverflow.com/questions/17935624/storagefile-50-times-slower-than-isolatedstoragefile/23213093 ) and i ended up with this approach (serializaton included) - see the question. – Vitalii Vasylenko May 10 '14 at 16:20
  • 1
    @VitaliiVasylenko Using methods from your EDIT3 is what I was writing about. There are possibilities to run the code from your question with Mutex - you should be able to do that by calling you Task.Factory.StartNew with LongRunning option, or simply Waiting on your jr.ReadJsonAsyncTask - as you run them on separate thread, there should be no problem. Thought the easies would be EDIT3. – Romasz May 10 '14 at 16:36
  • Just a simple addon to my question. What is a mutex name? It is said, name are implemented for a cross-thread access to the same mutex. It is a good idea to set a MutexName to a FileName? So mutexes (File1 and File2) in writers would be able to write into different files simultaneously, reader File3 would read file simultaneously with those two guys, but reader File2 would wait for a writer File2 to finish? – Vitalii Vasylenko May 10 '14 at 17:04
  • 1
    @VitaliiVasylenko When you specify MutexName then you are defining a global Mutex. The name should be unique for all the processes (not only yours), so simple filename can be too simple, but maybe try something specific for your app + fileName. Yes methods with different Mutexes will run simultanously and those with the same name will wait (of course if you want it to do so). – Romasz May 10 '14 at 17:11
  • Sorry, i'll uncheck this as solved, as i have some additional questions - maybe more ppl would join. – Vitalii Vasylenko May 11 '14 at 09:16
  • @VitaliiVasylenko Then better would be if you hadn't accepted the answer rather than accepting it, then changing the question along with unaccepting the answer. I've edited the answer maybe it will help. – Romasz May 11 '14 at 11:49
  • Yea, next time i'll do like that. What i to achieve is to have a fast standalone solution for async saving/loading of data (custom class) to/from IsoStorage. Particularly (that was original question), i want to have a mechanism to allow different writers/workers to work independent. Thanks for your sample, that seems pretty ok to me, i'll try it. However, this analysis ( http://stackoverflow.com/questions/17935624/storagefile-50-times-slower-than-isolatedstoragefile/23213093 ) shows that it would be faster to move reading and serializing to a separate subtask. That means i need awaitable mutex – Vitalii Vasylenko May 11 '14 at 13:50
  • Looking like files are corrupted in your case. Reader cant read them, and "Wp Power Tools" (i'm using it for reading isoStore data) also fails to open those files. – Vitalii Vasylenko May 11 '14 at 14:45
  • @VitaliiVasylenko Corrupted? - the method above is only opening files and reading them - I would better check the method that is saving those files - that one has an opportunity to corrupt. Shoot in the dark - check if you are opening file for writing or creating new. – Romasz May 11 '14 at 18:47
  • You're right: problem should be somewhere in the writing. Though, code should be fine: http://pastebin.com/4XLn38A3. Whole results are here: https://docs.google.com/spreadsheet/ccc?key=0AhmDOj13FFbgdFFIUHJhTU9XamR5NEFVLUlmb0NzZHc&usp=sharing – Vitalii Vasylenko May 11 '14 at 18:54
  • Well, i'll mark this as solved as i have a basic working version now (though, with AwaitableCriticalSection). – Vitalii Vasylenko May 11 '14 at 18:56
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/52475/discussion-between-romasz-and-vitalii-vasylenko) – Romasz May 11 '14 at 19:02