2

I'm troubleshooting a laundry list of exceptions that have recently begun to occur- We have a "single-use" service which we wrapped a bulk service around. The bulk service sets up all the variables required for each iteration, etc... The other variables are non-Entity/non-EF related. Roughly speaking:

    foreach (var item in ItemsToRun)
    {
        using (var context = new MyDbContext(connectionString))
        {
            var service = new Service(context, other, variables);
            
            var runResult = service.Run(item);
            
            results.add(runResult);
        }
    }

Each run gets a "fresh context". The individual runs of this service are unit tested end-to-end, so this really should have been a no-brainer. HOWEVER, after the 4th or sometimes 8 (but never more than 10) iterations, I'm getting a mixed bag of exceptions during the first SaveChanges call within the run.

Source: EntityFramework
Message: "Value cannot be null\r\n Parameter name: connection"
StackTrace: " at System.Data.Entity.Utilities.CheckNotNull[T](T value, String parameterName)"

Source: EntityFramework
Message: "Unable to cast object of type 'MyAssembly.Data.MyDbContext' to type 'System.Data.Entity.Core.Objects.ObjectContext'
StackTrace: " at System.Data.Entity.Infrastructure.TransactionHandler.get_ObjectContext()\r\n at System.Data.Entity.Infrastructure.TransactionHandler.Initialize(ObjectContext context)"

Source: System.Transactions
Message: "Unable to cast object of type 'System.Threading.Thread' to type 'System.Transactions.BucketSet'."
StackTrace: " at System.Transactions.TransactionTable.ThreadTimer(Object state)"

and a few others mentioning Transactions, ObjectContext, and SafeIUnknown.

It's worth noting that I had to use dotPeek to get a symbol file for EF to get the internal location of the Unable to cast to ObjectContext exception.

Things I have tried:

  • Connection pooling (at the connection string level was set to true by default, I've set it to false for this bulk operation) This seemed to have the biggest impact, as it wouldn't make it more than 2 runs with pooling enabled.
  • A delay 1 second after each run. No change in behavior.
  • A GC.Collect() after every 3rd run. Other than flushing out some bugs in an incorrectly implemented finalizer, no change in behavior.
  • A call to SqlConnection.ClearAllPools() after every run/3rd run. Immediately causes the next run to fail with one of the above exceptions.

Other notables:
This is being run within a MVVMLight WPF application.

Net Framework: 4.5

EF Version: EF 6.4.4

MS Sql Server.

MultipleActiveResultSets (MARS) is enabled.

The service is synchronous, and I can't find any location where an open read is clashing with a save changes call (such as iterating over an un-enumerated query result, whilst calling save changes within the loop). However, I would suspect that any outright terrible behavior like that would have been caught in unit testing, or result in all runs failing.

The SaveChanges() method is unaltered. We're not modifying EF's default transaction handling. Which from some quick googling, appears that the default behavior is to use a scoped transaction per SaveChanges() call.

I have a feeling the resources from the previous iteration are not being released correctly, but I'm unsure how to "force" that to happen beyond the contexts using statement and the things I've already tried.

Edit:

I have also tried overriding the SaveChanges() method in MyDbContext, conditionally wrapping the base.SaveChanges() as follows:


    public bool UseDbContextTransactions { get; set; }
    
    public override int SaveChanges()
    {
        var result = 0;
        
        if (UseDbContextTransactions)
        {
            try 
            {
                using (var dbContextTransaction = this.Database.BeginTransaction())
                {
                    result = base.SaveChanges();
                    dbContextTransaction.Commit();
                }
            }
            catch (Exception e)
            {
                System.Diagnostics.Debug.WriteLine($"Commit Exception: {e.message}");
                dbContextTransaction.RollBack();
            }
        }
        else
        {
            result = base.SaveChanges();
        }
        
        return result;
    }

From the try catch block, I now get the following exception:

Source: System.Transactions
Message: "Unable to cast object type 'System.Data.SqlClient.SqlTransaction' to type 'System.Transactions.SafeIUnknown'.
StackTrace: " at System.Transactions.Transaction.JitSaveGetContextTransaction(ContextData contextData)".

Andy Stagg
  • 373
  • 5
  • 22
  • Which RDBMS do you use? – Alexander Petrov Apr 28 '21 at 22:07
  • `service.Run` method is asynchronous? Does the method use multithreading internally? – Alexander Petrov Apr 28 '21 at 22:09
  • I updated the question with the Database provider, MS SQL Server in this case. service.Run is synchronous; not async. – Andy Stagg Apr 28 '21 at 22:11
  • @mjwills the code is non-sharable. The original question has the EF version, but it's 6.4.4. It was 6.4.1 prior to this issue, but we updated it with our fingers crossed. – Andy Stagg Apr 28 '21 at 23:49
  • What version of .NET Framework are you running? – mjwills Apr 29 '21 at 00:03
  • Added net framework version. 4.5 – Andy Stagg Apr 29 '21 at 00:07
  • Are you using WeakReference anywhere (https://stackoverflow.com/questions/67213321/weakreference-returns-wrong-object)? Or some form of SQL tracing (e.g. MiniProfiler)? – mjwills Apr 29 '21 at 02:24
  • Please share the [mcve] when you have created it. – mjwills Apr 29 '21 at 02:24
  • 1
    Do you have any static state that's preserved over iterations? And is EF configured to use proxies (which is the default)? Proxies that stay alive longer than one iteration may also cause the context to live longer than expected. You may want to try running with proxy creation disabled which won't make a difference if no lazy loading is involved. – Gert Arnold Apr 29 '21 at 07:05
  • No WeakReferences or SQL Tracing / Mini Profiler. We have lazy loading disabled, but I haven't heard of these proxies before. So I'll set that to false and cross my fingers. As for static state fields... The bulk service itself has one static field, and some of our data access layer (repos) have a handful of static fields which aren't EF related. But I put some console logging in the dispose method in the context, and can confirm it's being disposed of after each run. So I don't know if repos having a non EF static member would prevent the context from being properly garbage collected.. – Andy Stagg Apr 29 '21 at 14:50

1 Answers1

0

Thanks to @Gert Arnold's suggestion of disabling proxy creation, I was able to pinpoint that as the smoking gun which fixed the problem.

Working with proxies - EF 6 Microsoft Docs

I found that with proxy creation disabled, it was not necessary to override the SaveChanges() method with the Transaction wrapper. That should match the default EF behavior anyway.

For completeness and for those finding themselves here in the future:

You can disable proxy creation in the initializer of your DbContext class as such:

    public class MyDbContext() : DbContext
    {
        public MyDbContext(string connectionString) : base(connectionString)
        {
           Configuration.LazyLoadingEnabled = false;
           Configuration.ProxyCreationEnabled = false;
        }
        
        public DbSet<MyEntityOne> EntityOne { get; set; }
        public DbSet<MyEntityTwo> EntityTwo { get; set; }
    }

It's also worth noting, that I still found disabling connection pooling (via the connection string) to improve stability for the contexts used in the bulk service.

Andy Stagg
  • 373
  • 5
  • 22