1

One our customer has trouble with our application written in .NET. It consumes entire CPU after the start of the process without any windows to show. It looks like an infinite loop.

The problem is, that this customer has Windows 7 Home Edition and that is why I cannot connect with remote managed debugger to the application.

I can create special version with some debugging mode. I consider I could run a thread in the begging of the process and after about one minute call Debugger.Break and somehow log the threads and the stackframes. I am not sure if it is possible to log stackframes on all threads.

Do you have some experience with such kind of debugging without debugger attached?

UPDATE:

After two days of strugling I have finally found the problem. It was bad ATI Graphic Card driver, which caused the infinite loop. Because some compontents use OpenGL. The problem was easilly uncovered by iterating threads of the current process, suspending them a writting the stacktrace. The code I finally use to uncover the error is following:

using System;
using System.Diagnostics;
using System.Text;
using System.Threading;


    public class DebuggerContext
    {
        public int TimeIntervalMs { get; set; }
        public Thread InfiniteLoopThread { get; set; }
    }

    public static class InfiniteLogger
    {
        public static void LogInfniteLoopAfterTimeInterval(int timeIntervalMs)
        {
            Thread watchdog = new Thread(Watchdog);

            DebuggerContext context = new DebuggerContext()
            {
                InfiniteLoopThread = Thread.CurrentThread,
                TimeIntervalMs = timeIntervalMs,
            };

            watchdog.Start(context);
        }

        static void Watchdog(object threadContext)
        {
            DebuggerContext context = (DebuggerContext)threadContext;
            Thread.Sleep(context.TimeIntervalMs);

            LogThread(context.InfiniteLoopThread);

            var threads = Process.GetCurrentProcess().Threads;

            foreach (var thread in threads)
            {
                if (thread == Thread.CurrentThread)
                    continue;

                LogThread(context.InfiniteLoopThread);
            }
        }

        private static void LogThread(Thread thread)
        {
            thread.Suspend();           

            StackTrace stackTrace = new StackTrace(thread, true);

            string log = "The infinite loop is in: " + StackTraceToString(stackTrace);

            Trace.WriteLine(log);

            thread.Resume();
        }

        /// <summary>
        /// http://stackoverflow.com/questions/51768/print-stack-trace-information-from-c-sharp
        /// </summary>
        /// <returns></returns>
        static string StackTraceToString(StackTrace stackTrace)
        {
            StringBuilder sb = new StringBuilder(256);
            var frames = stackTrace.GetFrames();
            for (int i = 0; i < frames.Length; i++) /* Ignore current StackTraceToString method...*/
            {
                var currFrame = frames[i];
                var method = currFrame.GetMethod();
                var line = currFrame.GetFileLineNumber();
                var file = currFrame.GetFileName();
                var column = currFrame.GetFileColumnNumber();

                sb.AppendLine(string.Format("{0}:[{1},{2}], {3}.{4}",
                    file,
                    line,
                    column,
                    method.ReflectedType != null ? method.ReflectedType.Name : string.Empty,
                    method.Name
                    ));
            }
            return sb.ToString();
        }
    }

 [STAThread]
        static void Main(string[] args)
        {
            InfiniteLogger.LogInfniteLoopAfterTimeInterval(30 * 1000); //log thread state after 30 seconds after start
 ....
        }
Tomas Kubes
  • 23,880
  • 18
  • 111
  • 148

4 Answers4

4

If you have a list of all the running threads, then printing the stack frames is possible but not recommended.

StackTrace has a obsolete constructor -- StackTrace Constructor (Thread, Boolean).

So you could do something like:

foreach (var thread in list)
{
    thread.Suspend();
    Console.WriteLine(thread.ManagedThreadId);
    System.Diagnostics.StackTrace trace = new System.Diagnostics.StackTrace(thread, true);
    Console.WriteLine(trace);
}

Please read the Remark section from MSDN, because when you suspend a thread, you may cause deadlock...

Important note

Do not use this constructor. It is obsolete, and there is no recommended alternative. When you suspend a thread, you have no way of knowing what code it is executing, and deadlocks can occur very easily. For example, if you suspend a thread while it holds locks during a security permission evaluation, other threads in the AppDomain might be blocked. If you suspend a thread while it is executing a class constructor, other threads in the AppDomain that attempt to use that class are blocked.

I agree with others, you'd better go the log way rather than trying to suspend and print the stacks..

Community
  • 1
  • 1
terry
  • 1,569
  • 11
  • 19
  • I've had a problem on a web server, I've checked in the code for generating logs on to GitHub: https://github.com/Zaibot/ThreadWatchdog – Zaibot Mar 30 '15 at 20:59
1

In my opinion it is better start building a logging structure that write in windows log (or maybe a file) the instruction that your program is doing at a certain moment. You can make this as an option so you can make a production executable and use this type of logging on every client of yours.

Anyway, if you are interested in logging the stacktraces of all your threads, this answer may be useful for you Print thread stack of all threads of a process

Community
  • 1
  • 1
AlexF
  • 487
  • 1
  • 5
  • 16
1

Thank you for the tips. The application has it's own logging system which cover 99% of cases, but you cannot log everything. Finally I managed to do simple application to prove the infinite loop logging does work:

using System;
using System.Diagnostics;
using System.Text;
using System.Threading;

namespace InfiniteLoopDebug
{
    public class DebuggerContext
    {
        public int TimeIntervalMs { get; set; }
        public Thread InfiniteLoopThread { get; set; }
    }

    class Program
    {
        static void Main(string[] args)
        {
            LogInfniteLoopAfterTimeInterval(1200);

            WorkerMethod();
        }

        static void WorkerMethod()
        {

            int a = 0;
            int b = 1;
            while (true) //infinite loop
            {
                int c = a + b;
                b = a;            
            }
        }

        static void LogInfniteLoopAfterTimeInterval(int timeIntervalMs)
        {
            Thread watchdog = new Thread(Watchdog);

            DebuggerContext context = new DebuggerContext()
            {
                InfiniteLoopThread = Thread.CurrentThread,
                TimeIntervalMs = timeIntervalMs,
            };

            watchdog.Start(context);
        }

        static void Watchdog(object threadContext)
        {
            DebuggerContext context = (DebuggerContext)threadContext;
            Thread.Sleep(context.TimeIntervalMs);

            context.InfiniteLoopThread.Suspend();

            //Thread.Sleep(100);

            StackTrace stackTrace = new StackTrace(context.InfiniteLoopThread, true);

            string log = "The infinite loop is in: " +StackTraceToString(stackTrace);

            Trace.WriteLine( log);
            Console.WriteLine(log);

            context.InfiniteLoopThread.Resume();
        }

        /// <summary>
        /// http://stackoverflow.com/questions/51768/print-stack-trace-information-from-c-sharp
        /// </summary>
        /// <returns></returns>
        static public string StackTraceToString(StackTrace stackTrace)
        {
            StringBuilder sb = new StringBuilder(256);
            var frames = stackTrace.GetFrames();
            for (int i = 0; i < frames.Length; i++) /* Ignore current StackTraceToString method...*/
            {
                var currFrame = frames[i];
                var method = currFrame.GetMethod();
                var line = currFrame.GetFileLineNumber();
                var file = currFrame.GetFileName();
                var column = currFrame.GetFileColumnNumber();

                sb.AppendLine(string.Format("{0}:[{1},{2}], {3}.{4}",
                    file,
                    line,
                    column,
                    method.ReflectedType != null ? method.ReflectedType.Name : string.Empty,
                    method.Name
                    ));
            }
            return sb.ToString();
        }
    }
}
Tomas Kubes
  • 23,880
  • 18
  • 111
  • 148
0

While it might be feasible to do that I think that a better approach is to add logging to the application. Always add logging statements to your applications and a configurable way to enable logging when needed.

Use an existing logging framework (e.g. Log4Net) - they have options to enable logging in different steps.

Anders Abel
  • 67,989
  • 17
  • 150
  • 217