Flattening of AggregateExceptions for Processing

Question

I'm running into a few issues where I call flatten on an AggregateException, but inside there is still ANOTHER AggregateException! This obviously means that they are being propagated up the chain and being rolled into another AggregateException. Is there a way to recursively flatten ALL inner AggregateExceptions? Usually, I'll use the handle delegate to process these, but it returns false if there is another inner AggregateExceeption. Am I not handling these properly?

EDIT: Since I already am calling Flatten, it appears that the issue is that it's not being caught until way later in the callstack. Here is the code where I'm calling Flatten(). For use in the stack trace this method is called WriteExceptionRecord(string, FileInfo):

do
{
    try
    {
        using (var stream = file.Open(FileMode.Append, FileAccess.Write, FileShare.None))
        {
            using (StreamWriter writer = new StreamWriter(stream))
            {
                await writer.WriteLineAsync(data);
            }
        }
    }
    catch (AggregateException ex)
    {
        ex.Flatten().Handle((x) =>
        {
            if (x is IOException)
            {
                retryNeeded = true;
                retryLeft--;
                Thread.Sleep(500);
                return true;
            }

            logger.ErrorException("Could not write to exception file: " + data, ex);
            return false;
        });
    }
}
while (retryNeeded && retryLeft > 0);

However, the stack trace shows that it's not being caught here. Instead it's being caught way later up the call stack. Below is the trace with some identifying information removed for security reasons:

System.AggregateException: One or more errors occurred. --->      
System.AggregateException: One or more errors occurred. --->
System.IO.IOException: The process cannot access the file 'X:\Production\ProductionBatches\DataEntry\J\PD\Exception.csv' because it is being used by another process.    
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)    
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)    
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)    
   at System.IO.FileInfo.Open(FileMode mode, FileAccess access, FileShare share)    
   at PDI.LoadFileProcessing.<WriteExceptionRecord>d__21.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 328

--- End of stack trace from previous location where exception was thrown ---  
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at PDI.LoadFileProcessing.<ExceptionRecordProcessing>d__17.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 316

--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at PDI.ProcessPipeline.<>c__DisplayClass9.<<ProcessBatch>b__2>d__13.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\ProcessPipeline.cs:line 61

   --- End of inner exception stack trace ---

   --- End of inner exception stack trace ---

---> (Inner Exception #0) System.AggregateException: One or more errors occurred. ---> System.IO.IOException: The process cannot access the file 'X:\Production\ProductionBatches\DataEntry\J\PD\Exception.csv' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at System.IO.FileInfo.Open(FileMode mode, FileAccess access, FileShare share)
   at PeopleDocImporter.LoadFileProcessing.<WriteExceptionRecord>d__21.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 328

--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at PDI.LoadFileProcessing.<ExceptionRecordProcessing>d__17.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 316

--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at PDI.ProcessPipeline.<>c__DisplayClass9.<<ProcessBatch>b__2>d__13.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\ProcessPipeline.cs:line 61

   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.IO.IOException: The process cannot access the file 'X:\Production\ProductionBatches\DataEntry\J\PD\Exception.csv' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at System.IO.FileInfo.Open(FileMode mode, FileAccess access, FileShare share)
   at PDI.LoadFileProcessing.<WriteExceptionRecord>d__21.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 328

--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at PDI.LoadFileProcessing.<ExceptionRecordProcessing>d__17.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 316

--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()    
   at PDI.ProcessPipeline.<>c__DisplayClass9.<<ProcessBatch>b__2>d__13.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\ProcessPipeline.cs:line 61<---

<---

System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.IO.IOException: The process cannot access the file 'X:\Production\ProductionBatches\DataEntry\J\PD\Exception.csv' because it is being used by another process.    
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)    
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)    
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)    
   at System.IO.FileInfo.Open(FileMode mode, FileAccess access, FileShare share)    
   at PDI.LoadFileProcessing.<WriteExceptionRecord>d__21.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 328

--- End of stack trace from previous location where exception was thrown ---    
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()    
   at PDI.LoadFileProcessing.<ExceptionRecordProcessing>d__17.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 316

--- End of stack trace from previous location where exception was thrown ---    
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()    
   at PDI.ProcessPipeline.<>c__DisplayClass9.<<ProcessBatch>b__2>d__13.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\ProcessPipeline.cs:line 61

   --- End of inner exception stack trace ---

---> (Inner Exception #0) System.AggregateException: One or more errors occurred. ---> System.IO.IOException: The process cannot access the file 'X:\J\PD\Exception.csv' because it is being used by another process.    
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)    
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)    
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)    
   at System.IO.FileInfo.Open(FileMode mode, FileAccess access, FileShare share)    
   at PDI.LoadFileProcessing.<WriteExceptionRecord>d__21.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 328

--- End of stack trace from previous location where exception was thrown ---    
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()    
   at PDI.LoadFileProcessing.<ExceptionRecordProcessing>d__17.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 316

--- End of stack trace from previous location where exception was thrown ---    
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()    
   at PDI.ProcessPipeline.<>c__DisplayClass9.<<ProcessBatch>b__2>d__13.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\ProcessPipeline.cs:line 61

   --- End of inner exception stack trace ---    
---> (Inner Exception #0) System.IO.IOException: The process cannot access the file 'X:\Production\ProductionBatches\DataEntry\J\PD\Exception.csv' because it is being used by another process.    
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)    
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)    
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)    
   at System.IO.FileInfo.Open(FileMode mode, FileAccess access, FileShare share)    
   at PDI.LoadFileProcessing.<WriteExceptionRecord>d__21.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 328

--- End of stack trace from previous location where exception was thrown ---    
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()    
   at PDI.LoadFileProcessing.<ExceptionRecordProcessing>d__17.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\LoadFileProcessing.cs:line 316

--- End of stack trace from previous location where exception was thrown ---    
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()    
   at PDI.ProcessPipeline.<>c__DisplayClass9.<<ProcessBatch>b__2>d__13.MoveNext() in c:\Users\XYZ\Development\PDI\PDI\ProcessPipeline.cs:line 61<---

By the way: This is being called by TPL-Dataflow blocks.

It's just a standard handle that returns true if it's an IOException and false otherwise. I was thinking about writing a `Func` recursively, but not sure if this is the correct behavior. — JNYRanger, Apr 04 '14 at 21:14
Yes. I have them showing up in Log2Console, but still being thrown when I attempt to catch them. — JNYRanger, Apr 04 '14 at 21:20
@KB Probably too many, but it's a lot. I can post some sample code tomorrow, but going to do some more testing first. — JNYRanger, Apr 06 '14 at 00:39
@KB added the code & the stack trace with the inner AggregateException — JNYRanger, Apr 07 '14 at 14:57
@JNYRanger, you main problem looks like a typo to me. You are logging `ex`, which is the AggregateException you caught, instead of `x`, which is the inner exception passed to the lambda. — Palec, Feb 16 '22 at 15:15

score 38 · Answer 1 · answered Oct 13 '17 at 15:01

Keep in mind that the 'flatten' method will give you a list of Exceptions, but can still leave you with a flattened InnerExceptions inside of each Exception.

So I found that this was not really sufficient:

try
{
    // something dangerous
}
catch (AggregateException ae)
{ 
    foreach(Exception innerException in ae.Flatten().InnerExceptions)
    {
        Console.WriteLine(innerException.Message());
    }
}

Because this exception:

System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 192.168.42.55:443 at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult) at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception) --- End of inner exception stack trace --- at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context) at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar) --- End of inner exception stack trace ---

Would end up like this:

An error occurred while sending the request.

The fix was something like this:

foreach(Exception exInnerException in aggEx.Flatten().InnerExceptions)
{
    Exception exNestedInnerException = exInnerException;
    do
    {
        if (!string.IsNullOrEmpty(exNestedInnerException.Message))
        {
            Console.WriteLine(exNestedInnerException.Message);
        }
        exNestedInnerException = exNestedInnerException.InnerException;
    }
    while (exNestedInnerException != null);
}

Resulting in:

An error occurred while sending the request.

Unable to connect to the remote server

A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 192.168.42.54:443

Hope that helps someone.

To clarify, the Flatten method flattens AggregateExceptions recursively. It does not flatten normal exceptions that have a single inner-exception. In other words, it unpacks all AggregateExceptions recursively, looking at their InnerExceptions (plural) property and so on. It does not unpack the InnerException (singular) property of non-aggregate exceptions. You'll have to do that on your own after calling Flatten on the top-level AggregateException. This is the most accurate answer. — Triynko, Oct 11 '18 at 16:29

score 30 · Answer 2 · edited Mar 11 '16 at 07:43

30

Yes, there's exactly what you're asking for:

AggreggateException.Flatten()

will go through and compress everything down to a single AggregateException. So you can use it to loop through the all the inner exceptions like this:

try
{
    // something dangerous
}
catch (AggregateException ae)
{ 
    foreach(var innerException in ae.Flatten().InnerExceptions)
    {
        // handle error
    }
}

MSDN link: http://msdn.microsoft.com/en-us/library/system.aggregateexception.flatten.aspx

edited Mar 11 '16 at 07:43

John

6,503
3
37
58

answered Apr 04 '14 at 21:16

Sean U

6,730
1
24
43

Hm..maybe I'm doing something wrong then, because that's what I'm doing. – JNYRanger Apr 04 '14 at 21:20
@JNYRanger Added some more sample code. If that doesn't help, could you describe what the tree of exceptions looks like and the results `.Flatten()` is giving you in more detail? – Sean U Apr 04 '14 at 21:27
I'll post some sample code tomorrow, going to do a bit of testing first tomorrow though. – JNYRanger Apr 06 '14 at 00:40

SensorSmith · Answer 3 · 2017-08-29T15:57:30.917

This is an old question, but the problem experienced by the OP is that await does not expose the AggregateException from the awaited task, but instead just the first exception in the AggregateException. So the catch(AggregateException ex) block is bypassed and the exception is caught further up the stack. So the code should have been 'simply':

retryNeeded = false;
do
{
    try
    {
        if (retryNeeded)
            await Task.Delay(500); // substituted for Thread.Sleep

        using (var stream = file.Open(FileMode.Append, FileAccess.Write, FileShare.None))
        {
            using (StreamWriter writer = new StreamWriter(stream))
            {
                await writer.WriteLineAsync(data);
                retryNeeded = false;
            }
        }
    }
    catch (IOException)
    {
        retryNeeded = true;
        retryLeft--;
    }
    catch (Exception ex)
    {
        logger.ErrorException("Could not write to exception file: " + data, ex);
        throw;
    }
} while (retryNeeded && retryLeft > 0);

return (retryLeft > 0);

Alternatively, Jon Skeet's WithAllExceptions extension method allows one to 'protect' the AggregateException from await's behavior by wrapping the task in another task so you get an AggregateException containing an AggregateException and await 'returns' the original/inner AggregateException.

NOTE: AggregateException.Flatten really does 'flatten' recursively, as is shown by the example on the MSDN page.

EDIT: Improved delay on retryNeeded to avoid setting a bad async example.

score 4 · Answer 4 · answered Apr 04 '14 at 21:21

Normally AggregateException is used to consolidate multiple failures into a single, throwable exception object.

try {
          Task.WaitAll(tasks)
      }
      catch (AggregateException ae) {
          ae.Handle((x) =>
          {
              if (x is UnauthorizedAccessException) // This we know how to handle.
              {
                 //do your code here  
              }
               return true; //if you do something like this all exceptions are marked as handled  
           });
      }

score 0 · Answer 5 · answered Oct 13 '17 at 15:22

Try the sample code below, this should avoid await unwrapping the AggregateException and throw the original AggregateException at the point where task.result is called.

var task = writer.WriteLineAsync(data);
            await task.ContinueWith(t => { }, TaskContinuationOptions.ExecuteSynchronously));
            return task.Result;

Flattening of AggregateExceptions for Processing

5 Answers5

Linked