Can I detect a hung process with another thread and recover from it?

Question

I have a method that occasionally hangs (in a dll I cannot modify but must use). If I run it again It will typically work fine. I was wondering if it would be possible to make a background thread that would wait for 20 minutes and then throw an exception in my program.

var triesLeft = 5;
while (triesLeft > 0) {
  try {
    var t = new Thread(() => { wait(20 minutes); throw new ApplicationHungException();})
    t.Start();
    Object o = MethodThatHangsForever10PercentOfTheTime();
  } catch (ApplicationHungException e) {
    triesLeft--;
  }
}
t.Abort();

This does not work because the exception does not pass to the try catch block it's contained in. Is there a way I can get the thread to give it's exception to the try catch block?

FYI, there's no such thing as a "C# thread". You appear to be referring to .NET managed threads. — John Saunders, Oct 11 '13 at 23:13
Why do you need thread here, just place code inplace of "methodthathands" - than you will catch exception — Iłya Bursov, Oct 11 '13 at 23:14
@IlyaBursov `MethodThatHangsForever` comes from the DLL I am assuming. — Asad Saeeduddin, Oct 11 '13 at 23:14
@anthonybell Would it be feasible to spawn a thread to run `MethodThatHangsForever`, and wait for a predeterminate amount of time in the main thread, then check whether the other thread is still alive? Essentially inverting which thread runs the faulty method and which thread is resposible for sounding the alarm. — Asad Saeeduddin, Oct 11 '13 at 23:18
See [this question](http://stackoverflow.com/questions/2335238/calling-thread-abort-on-a-thread-from-a-threadpool) for someone in a very similar situation. Apparently the optimal solution is to use a separate process, and not just a separate thread. — Asad Saeeduddin, Oct 11 '13 at 23:51

Asad Saeeduddin · Answer 1 · 2013-10-11T23:58:33.360

3

One way to do this would be set off your faulty method in the separate thread, and wait for one of two things to happen; either:

The thread completes, or
A predetermined amount of time (eg 20 mins) elapses

Once either of these things happens, we can take appropriate action.

The code would look something like this:

static void DoProcessing() {
    var triesLeft = 5;
    Object o = null;

    while (triesLeft > 0) {
        var t = new Thread(() => { o = MethodThatHangsForever10%OfTheTime(); }).Start();
        if (t.Join(new TimeSpan(0, 20, 0))) {
            // The thread completed
            break;
        } else {
            // We're out of time.
            t.Abort(); // Important: See comments below about this
            triesLeft--;
        }
    }
}

It turns out that aborting threads is a risky and fragile operation, as pointed out by Reed Copsey in the comments below. Your alternatives are to allow the hung thread to live out the rest of its life (however long that may be), or to quarantine the Heisenbuggy method call to a separate process.

This opens up another can of worms however, since you would have to deal with interprocess communication, data serialization and synchronisation. This may or may not be worth it, which is a judgement call I leave to you.

edited Oct 11 '13 at 23:58

answered Oct 11 '13 at 23:33

Asad Saeeduddin

46,193
6
90
139

Thread.Abort is really not a very good idea - See http://stackoverflow.com/q/1559255/65358 – Reed Copsey Oct 11 '13 at 23:42
@ReedCopsey I see. Is it a better idea to just let the thread live out the remainder of its (possibly permanent) existence? Or is there a better way to limit the lifespan of the thread? I don't have a lot of experience with multithreading, so bear with me please. – Asad Saeeduddin Oct 11 '13 at 23:46
It really depends - in general, it'd be best to fix the problem (prevent the infinite hang), but Thread.Abort is often worse than letting the thread sit there forever... it can cause nasty problems (see the many answers in that question) – Reed Copsey Oct 11 '13 at 23:51
If this ran every 10 minutes for a few days wouldn't my service eventually be filled with threads doing nothing if I don't use abort()? – anthonybell Oct 15 '13 at 18:21
@anthonybell Yes, which is why I mentioned the alternative. I believe Reed's answer suffers from the same problem. – Asad Saeeduddin Oct 15 '13 at 20:38

Reed Copsey · Accepted Answer · 2013-10-11T23:41:55.157

2

You can do your work in a separate thread, and wait 20 minutes for it to complete:

var triesLeft = 5;
while (triesLeft > 0) 
{
    var mre = new ManualResetEvent(false);

    ThreadPool.QueueUserWorkItem(_ => {
                           MethodThatHangsForever10PercentOfTheTime();
                           mre.Set();
                     });

    if (mre.WaitOne(TimeSpan.FromMinutes(20)))
    {
          break; // Success!
    }
    triesLeft--;
  }
}

edited Oct 11 '13 at 23:41

answered Oct 11 '13 at 23:28

Reed Copsey

554,122
78
1,158
1,373

Will this always wait 20 minutes or does the mre.Set() cause it to break out of the while loop? – anthonybell Oct 11 '13 at 23:33
@anthonybell The `mre.Set()` will cause it to break out immediately. – Reed Copsey Oct 11 '13 at 23:37
1

@anthonybell The 20 minutes is the most it'll wait, then it'll start over – Reed Copsey Oct 11 '13 at 23:38
@anthonybell Edited to use the ThreadPool, which is likely cleaner for you, too – Reed Copsey Oct 11 '13 at 23:43

Can I detect a hung process with another thread and recover from it?

2 Answers2