2

Application exists unexpectedly during iterating over IAsyncEnumerable and inserting records to database using EF context. No error or exception is thrown. In the console window it shows only The program '[5372] XYZ.exe' has exited with code 3 (0x3).

The code is somthing like:

public async Task LoadDataAsync()
{
    try
    {
        _context.ChangeTracker.AutoDetectChangesEnabled = false;
        int bufferSize = 50;
        var myEntityBuffer = new List<MyEntity>(bufferSize);

        //GetDataAsync returns IAsyncEnumerable
        await foreach (var element in loader.GetDataAsync(cancellationToken).WithCancellation(cancellationToken))
        {
            myEntityBuffer.Add(new MyEntity(element));

            if (addressPointBuffer.Count == bufferSize)
            {
                _context.MyEntities.AddRange(myEntityBuffer);
                await _context.SaveChangesAsync(cancellationToken);
                myEntityBuffer = new List<MyEntity>(bufferSize);
                _context.ChangeTracker.Clear();
            }
        }

        await _context.SaveChangesAsync(cancellationToken);
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, ex.Message);
    }
    finally
    {
        _context.ChangeTracker.AutoDetectChangesEnabled = true;
    }
}

The same pattern is used by other commands and they work fine, I could not spot the difference except that the element structure is different, different entity. The number of records is large, but the application exists after inserting approx. 80 000 record (the number differs from run to run)

I cannot trace the source of the problem and what makes the application exit. I run this on Development environment.

I appreciate if you could suggest how to trace the issue

So far I have tried the following:

  • Checking the application Logs - the running code is wrapped with try-catch, however no error were thrown and logged
  • placing a breakpoint in catch and finally blocks (these lines are not reached)
  • Wrapping Main method code in try-catch (no execption was catched)
  • Adding IHostApplicationLifetime Stopped and OnStopping events (these events are not raised upon exit)
  • increasing logging level for Microsoft.EntityFramework to Debug (I can see only SQL commands but no issues)

If I comment out lines doing database operations (AddRange, SaveChangesAsync) the method completes without issues and the application is still running.

I use the following stack

  • .NET Runtime 5.0.11
  • ASP.NET Core Runtime 5.0.11
  • .NET Desktop Runtime 5.0.11
  • EntityFramework Core + NetTopologySuite
  • SQLite + Spatialite

UPDATE

I have stopped using SQLite. I had plans to move to postgresql anyway. On postgresql it does not happen. Still would be nice to know how to diagnose such issues...

Sebastian Widz
  • 1,962
  • 4
  • 26
  • 45
  • According to microsoft doc: https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499- , Code 3 is related to : "The system cannot find the path specified." – T.Trassoudaine Oct 20 '21 at 12:39
  • Some sources suggest that this may be caused by the system running out of memory. And the fact that the application exits after you inserted a significant amount of rows may confirm this theory. – T.Trassoudaine Oct 20 '21 at 12:59
  • It does not seem it is related to out of memory issue. When it happens, server process takes less than 3Gig while there are still available resources. – Sebastian Widz Oct 20 '21 at 13:58
  • Well make sure you don't exceed maximum memory size for a process and for any of your lists (such as `MyEntities`). According to this source: https://stackoverflow.com/questions/53193085/memory-limitted-to-about-2-5-gb-for-single-net-process , the limit for arrays is *2GB*, I don't know if it changed. – T.Trassoudaine Oct 20 '21 at 14:10
  • 2
    1) Clear ChangeTracler after SaveChanges `_context.ChangeTracker.Clear()` 2) ensure that you have not executed LoadDataAsync in multiple threads, eg. Task.WaitAll 3) Consider using bulk extensions, 10000 records is a second to insert. – Svyatoslav Danyliv Oct 20 '21 at 15:06
  • @SvyatoslavDanyliv. I was also trying to use _context.ChangeTracker.Clear() - same issue, but agree It should be included. I definitely need to look at Bulk extensions. One obstacle is that I have spacial extensions (geometry) + SQLite on DEV and planned to use PostgreSQL on Prod. – Sebastian Widz Oct 20 '21 at 21:57

0 Answers0