0

I am running a BackgroundTask in my ASP.NET Core 5 app which creates some entities inside my database. My idea was to run it in multiple tasks to speed up the generator.

    public class GeneratorService : BackgroundService
    {
        private const int TaskCount = 8;

        private uint _index;
        private readonly List<Task> _tasks;
        private readonly IServiceScopeFactory _serviceScopeFactory;

        public GeneratorService(IServiceScopeFactory serviceScopeFactory)
        {
            _serviceScopeFactory = serviceScopeFactory;

            _tasks = new List<Task>();
        }

        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            while (stoppingToken.IsCancellationRequested == false)
            {
                for (uint i = 0; i < TaskCount; i++)
                {
                    var i1 = i;

                    var task = new Task(async () => await Generate(_index + i1, stoppingToken));

                    task.Start();
                    _tasks.Add(task);
                }

                Task.WaitAll(_tasks.ToArray(), stoppingToken);

                _tasks.Clear();

                _index += TaskCount;
                await Task.Delay(10, stoppingToken);
            }
        }

        private async Task Generate(uint index, CancellationToken stoppingToken)
        {
            using var scope = _serviceScopeFactory.CreateScope();

            var context = scope.ServiceProvider.GetService<DataContext>();
            var repository = scope.ServiceProvider.GetService<Repository>();
            var generator = scope.ServiceProvider.GetService<Generator>();

            // A simple return await FirstOrDefaultAsync(e => e.Number == index)
            var entity = await repository.GetByNumberAsync(index);
            if (entity== null) ...
                

            generator.GenerateChildren(entity);

            await context.SaveChangesAsync(stoppingToken);

            Console.WriteLine($"Entity {index} generated");
        }
    }

The generator loop runs for about 100 - 200 entities, but then it starts throwing me exceptions:

fail: Microsoft.EntityFrameworkCore.Query[10100]
      An exception occurred while iterating over the results of a query for context type 'Namespace.DataContext'.
      Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
       ---> System.ComponentModel.Win32Exception (258): Der Wartevorgang wurde abgebrochen.
         at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__169_0(Task`1 result)
         at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
         at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
      --- End of stack trace from previous location ---
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(DbContext _, Boolean result, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
      ClientConnectionId:f8508f09-1298-487c-8513-93bd4289014e
      Error Number:-2,State:0,Class:11
      Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
       ---> System.ComponentModel.Win32Exception (258): Der Wartevorgang wurde abgebrochen.
         at Microsoft.Data.SqlClient.SqlCommand.<>c.<ExecuteDbDataReaderAsync>b__169_0(Task`1 result)
         at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
         at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
      --- End of stack trace from previous location ---
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(DbContext _, Boolean result, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementationAsync[TState,TResult](Func`4 operation, Func`4 verifySucceeded, TState state, CancellationToken cancellationToken)
         at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
      ClientConnectionId:f8508f09-1298-487c-8513-93bd4289014e
      Error Number:-2,State:0,Class:11

I am registering the DataContext in an extension method:

        public static IServiceCollection ConfigureDatabase(this IServiceCollection services,
            IConfiguration configuration)
        {
            
            var connectionString = // I get my connection string via a json file

            services.AddDbContext<DataContext>(options =>
            {
                options.UseSqlServer(connectionString);
                options.EnableSensitiveDataLogging();
            });

            return services;
        }

This is a SQL Server Database running on my localhost. If I am right, the DataContext is registered as a Scoped Service. So if I am creating a scope inside the BackgroundTask, it will automatically dispose the DataContext after the Task completed.

Or is this a SQL Server related issue where it could not handle that many requests simultaneously?

Dale K
  • 25,246
  • 15
  • 42
  • 71
DirtyNative
  • 2,553
  • 2
  • 33
  • 58
  • I see some similarities with this recent question: [await Task does not wait](https://stackoverflow.com/questions/66644454/await-task-does-not-wait). Task constructor with `async void` lambda = problems. – Theodor Zoulias Mar 17 '21 at 01:31
  • Good point, I changed the line to ``` new Task(...) ``` but it didn't solve the problem – DirtyNative Mar 17 '21 at 01:38
  • Instead of `var task = new Task(async`, it's better `var task = Task.Run(async`, and then remove the `task.Start()` afterwards. Cold tasks and nested tasks shouldn't be really used in application code. They have some rare uses in library code. – Theodor Zoulias Mar 17 '21 at 02:21
  • 1
    Omitting the `Task.Run` is probably also an option: `var task = Generate(_index + i1, stoppingToken);`. This actually may be the best option. – Theodor Zoulias Mar 17 '21 at 02:23
  • I think this is almost obviously an issue of the DB Server, it cannot handle many requests at the same time and the processing time is increased, making the timedout exceptions arise. So try increasing the timeout (default by 30 seconds) of the command execution to see if it's the issue. – King King Mar 17 '21 at 03:56
  • How is your repository resolving the scoped instance of the DbContext? This implementation is heavily dependent on a service locator pattern. It has it's uses, I use one for repository patterns called the DbContextScope from Mehdime, which has a context scope factory and an ambient scope locator, but I don't recommend having a Service Locator to replace dependency injection. – Steve Py Mar 17 '21 at 04:09
  • 1
    @TheodorZoulias Instanting the Task w/o starting it solves the exceptions. I don't know why, but at least i was able to run Generate() 4000 times without any error – DirtyNative Mar 17 '21 at 10:13

0 Answers0