Pending tasks pattern: try it later pattern

Question

I'm using several FileSystemWatcher objects in order to watch several directories out.

When I do a Copy&Paste operation of several files on a folder which's watched out, the FileSystemWatcher raises an Created event for each new created file, which I capture.

I'm using a Thread Pool with a concurrency of n in order to enqueue jobs and control how many threads I want are working on it (I could create 200 files at once, however, I only want 4 threads works on it). "It" means to calculate a checksum:

private byte[] calculateChecksum(string frl)
{
    byte[] checksum = null;
    FileStream stream = null;
    try
    {
        stream = File.Open(frl, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
        MD5 md5 = MD5.Create();
        checksum = md5.ComputeHash(stream);

    }
    finally
    {
        stream.Close();
    }

    return checksum;
}

The problem is that sometimes when a thread is working on that job (calculateChecksum method), it tries to get access to file, however, sometimes the file is in use (I suppouse by the SO).

I want to set a solution in order to solve this telling me: "try it later".

One solution would be, enqueue again the job on the thread pool. The problem is that I don't know how to tell later. I believe it's important, because I don't want it breaths and rest a short time and tries it again later.

For example:

New file created on watched out folder.
FileSystemWatcher raises an event.
I enqueue the job in the thread pool.
The job crashes beacause the file I want to get access is in use by another process.
enqueue again the job and wait 60 seconds to try it again
60 seconds later, the job crashes again, I want the job is enqueued again and waits 300 seconds.
300 seconds later, it tries again and it has been able to calculate the checksum correctly.

I don't want that: wait 300 seconds means a thread of the thread pool is busy. It would not allow to the other enqueued jobs be dispatched.

I don't know if I've been clear. Any ideas

You can use `TaskCompletionSource` if you want your thread to perform other tasks in case the file is held by another process. But this will require you to use async / await all over your code - which is not a bad thing. — shay__, Dec 07 '15 at 13:16
You say "wait 300 seconds means a thread of the thread pool is busy". That is not true. Wait with `await Task.Delay(300000);`. — Kris Vandermotten, Dec 07 '15 at 13:22

score 2 · Answer 1 · edited May 23 '17 at 11:59

2

Basically you want a retry pattern with timer.

You can do that easily with Polly:

// Retry, waiting a specified duration between each retry
Policy
  .Handle<DivideByZeroException>()
  .WaitAndRetry(new[]
  {
    TimeSpan.FromSeconds(1),
    TimeSpan.FromSeconds(2),
    TimeSpan.FromSeconds(3)
  });

https://github.com/App-vNext/Polly

or like the follwing without polly:

https://stackoverflow.com/a/1563234/1384539

edited May 23 '17 at 11:59

Community

1
1

answered Dec 07 '15 at 13:16

Thiago Custodio

17,332
6
45
90

Ok. That's really awesome! I've figured out I'm able to set a behavior policy. However, I can set a behavior for retring up to _n_ times, for example: "right now", 120 seconds later, and a last one 300 seconds later. This behavior policy takes a thread of my pool busy along 120 + 300 = 420 seconds. Am I wrong? – Jordi Dec 07 '15 at 21:12
you can, using a Task with timeout cancelation. https://msdn.microsoft.com/en-us/library/dd321457%28v=vs.110%29.aspx – Thiago Custodio Dec 08 '15 at 10:50

Pending tasks pattern: try it later pattern

1 Answers1