155

I am running some correction code that runs over a big pile of entities, as it progress its speed decreases, that is because the number of tracked entities in the context increase with each iteration, It can take long so I am saving changes at the end of each iteration. Each iteration is independent and does not change the previosuly loaded entities.

I know I can turn off change tracking but I do not want to, because it is not a bulk insert code, but loading the entities and calculating a few things and if the numbers are not correct set the new numbers and update/delete/create some additional entities. I know I can create a new DbContext for each iteration and probably that would run faster than doing all in the same instance, but I am thinking that there might be a better way.

So the question is; Is there a way of clearing the entities previously loaded in the db context?

Sarath Subramanian
  • 20,027
  • 11
  • 82
  • 86
hazimdikenli
  • 5,709
  • 8
  • 37
  • 67
  • 18
    You can just call `context.Entry(entity).State = EntityState.Detached` and it will stop tracking that particular entity. – Ben Robinson Dec 11 '14 at 12:41
  • 4
    Why don't you just instantiate a new Context? There is really no big overhead unless you need very optimized code. – Adrian Nasui Dec 11 '14 at 12:42
  • entity framework hits the database server only for the changed entities, you don't have no performance concerns about that. but you can create a new context only consisting of the tables you work with to make it faster. – İsmet Alkan Dec 11 '14 at 12:44
  • 1
    @IsThatSo detecting the changes take time, I am not worried about DbPerformance. – hazimdikenli Dec 11 '14 at 12:48
  • have you actually debugged and tracked the performance bottleneck, or just assuming this? – İsmet Alkan Dec 11 '14 at 12:49

6 Answers6

178

Nowadays, you can just call context.ChangeTracker.Clear(); if you're using EF Core 5.0+


Otherwise, you can add a method to your DbContext or an extension method that uses the ChangeTracker to detach all the Added, Modified, Deleted, and Changed entities:

public void DetachAllEntities()
{
    var undetachedEntriesCopy = this.ChangeTracker.Entries()
        .Where(e => e.State != EntityState.Detached)
        .ToList();

    foreach (var entry in undetachedEntriesCopy)
        entry.State = EntityState.Detached;
}
David Sherret
  • 101,669
  • 28
  • 188
  • 178
  • 6
    Make sure to call 'ToList' after the 'Where'. Otherwise, it throws a System.InvalidOperationException: 'Collection was modified; enumeration operation may not execute.' – mabead Jul 19 '17 at 18:19
  • @mabead interesting... maybe that happens in newer versions of EF. I've updated the answer to include your suggestion because it seems like a good thing to do just in case. Thanks! – David Sherret Jul 19 '17 at 18:23
  • 7
    In my unit test, the entries state is "Unmodified", maybse because I use a transaction that I roll back at the end of the test method. This meant that I had to set tracked entries state to "Detached" without checking current state, so that my tests run all correctly at once. I call the above code right after rolling back transaction, but I got it, rolling back surely means Unmodified state. – barbara.post Oct 10 '17 at 12:16
  • the explicit foreach loop can even be embedded directly in the lambda expression right after converting to list: .ForEach(e => e.State = EntityState.Detached); – PillowMetal Aug 02 '18 at 23:49
  • Do you need to do `this.Entry(entity.Entity).State = Detached`? I think you can just do `entity.State = Detached` See [this answer](https://stackoverflow.com/a/42885424/868159) – oatsoda Aug 28 '18 at 10:56
  • 2
    (and also `var entity` should really be `var entry` as it is the entry not the actual Entity) – oatsoda Aug 28 '18 at 10:58
  • 1
    @OffHeGoes yes, you're right. Thanks! There was some gymnastics going on there due to the poor naming :) – David Sherret Aug 28 '18 at 13:07
  • 2
    @DavidSherret Thought that might be the case! I caught this because in a test app of mine, cycling through 1000 items and marking as Detached took approx 6000ms with the existing code. About 15ms with the new :) – oatsoda Aug 28 '18 at 13:22
  • 7
    Shouldn't you also use e.State == EntityState.Unchanged? Although the entity is unchanged, it is still tracked in the context and is a part of set of entities that are considered during DetectChanges. E.g. you add new entity (it is in state Added), call SaveChanges and the added entity has now state Unchanged (it goes against the UnitOfWork pattern, but op asked: *I am saving changes at the end of each iteration*). – jahav Mar 19 '19 at 13:31
  • 1
    How does setting an entry's state to "Detatched" remove it from the change tracker. Seems like it's still tracked as an entry, just in a Detached state. Won't this still allow item in memory to grow? – Triynko Oct 23 '19 at 18:15
  • 1
    @Triynko It's still cached and consuming memory, but the change tracker will ignore detached entities so future operations run faster. – Suncat2000 Jan 02 '20 at 18:34
  • 1
    @jahav Not adding EntityState.Unchanged will cause EF Core to throw this exception in some scenarios "The instance of entity type 'AAA' cannot be tracked because another instance with the key value '{AAA: 20010}' is already being tracked. ". In my scenario, I am retrieving entity AAA in one query, do some updates, call `SaveChanges`, and then I make a separate update (within the same scope of the DI container) to this entity, which fails with the above error, due to this instance already being tracked. – Adrian Jan 07 '21 at 12:28
  • Some entities may still be tracked with the state "Unchanged". You may want to consider clearing these as well. – Andreas Olesen Feb 02 '21 at 12:56
  • This doesn't appear to actually work. With a "completely clear context", removing an entity will still trigger relationship severing failures. – Captain Prinny Jun 04 '21 at 15:55
  • It would be a bit more efficient if you cast it `.AsEnumerable()` no? – MikeT Jun 09 '21 at 18:08
  • 1
    @MichaelTranchida I think the collection changes while enumerating so that's why the copy is done (to prevent a "collection changed while enumerating" exception). Edit: Yes, that's it. See the first comment on this answer. – David Sherret Jun 09 '21 at 18:10
  • This solution doesn't cover the state Unchanged. With `_context.ChangeTracker.Clear();` it cleans all the tracking items. – Pablo Caballero Mar 20 '23 at 16:51
141

EntityFramework Core 5.0 introduced a new method to clear any tracked changes.

_context.ChangeTracker.Clear();

https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.changetracking.changetracker.clear?view=efcore-5.0

j-petty
  • 2,668
  • 1
  • 11
  • 20
  • 6
    Worth pointing out, this is also the better answer performance wise, too. Per the documentation: "This method should always be preferred over detaching every tracked entity. Detaching entities is a slow process that may have side effects. This method is much more efficient at clearing all tracked entities from the context." – schil227 Nov 11 '21 at 15:08
  • Hi, It works well thanks about it. I solved my problem and I understand why thank you. One more problem I have, If ef cant add records to db, it should write to log. But it wasnt. It was the first problem, because he tries to add all tracked entities, I deleted for problem record then I added just log record to entity, Then it saved well. But still it not throws an error. I cant debuging because it goes to threadpool.cs or some files like that if I want to debug more it wants to download some files, actually I did it downloaded many files about threads but nothing, I didnt see anything – Çağlar Can Sarıkaya Dec 10 '21 at 14:00
  • 11
    This should be the accepted answer in 2022. – Anders Marzi Tornblad Feb 10 '22 at 14:07
34

1. Possibility: detach the entry

dbContext.Entry(entity).State = EntityState.Detached;

When you detach the entry the change tracker will stop tracking it (and should result in better performance)

See: http://msdn.microsoft.com/de-de/library/system.data.entitystate(v=vs.110).aspx

2. Possibility: work with your own Status field + disconnected contexts

Maybe you want to control the status of your entity independently so you can use disconnected graphs. Add a property for the entity status and transform this status into the dbContext.Entry(entity).State when performing operations (use a repository to do this)

public class Foo
{
    public EntityStatus EntityStatus { get; set; }
}

public enum EntityStatus
{
    Unmodified,
    Modified,
    Added
}

See following link for an example: https://www.safaribooksonline.com/library/view/programming-entity-framework/9781449331825/ch04s06.html

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
road242
  • 2,492
  • 22
  • 30
  • I think adding an extension method and running over all entities in the ChangeTracker and detaching them should work. – hazimdikenli Dec 11 '14 at 13:07
20

I'm running a windows service that updates values every minute and I have had the same problem. I tried running @DavidSherrets solution but after a few hours this got slow as well. My solution was to simply create a new context like this for every new run. Simple but it works.

_dbContext = new DbContext();

Ogglas
  • 62,132
  • 37
  • 328
  • 418
  • 8
    This is not 'Simple but works' solution for your goal. It is the only correct one. Context should live as less as possible, 1 context per 1 transaction is the best option. – Yehor Androsov Jul 30 '18 at 12:21
  • 2
    Agreed with @pwrigshihanomoronimo, the context follows the UnitOfWork design pattern. As defined by Martin Fowler: > Maintains a list of objects affected by a business transaction and > coordinates the writing out of changes and the resolution of concurrency > problems. – Michel Aug 30 '18 at 07:24
  • This seemed to do the trick for me. I'm synchronizing data, having approximately halv million transactions (insert and update) in tables with a couple of million rows. So I struggled with OutOfMemoryException after a while (or a number of operations). This resolved when I created a new DbContext per X number of loops that was a natural place to re-instance the context. That probably triggered GC in a better way, not having to think about the possible memory leak in EF and long running operations. Thank you! – Mats Magnem Aug 14 '20 at 04:55
  • 2
    not sure if this works with dependency injection – Sabrina Leggett Dec 14 '20 at 19:40
  • @SabrinaLeggett It should work no matter where the context was originated from – Ogglas Dec 14 '20 at 19:48
  • 1
    @MatsMagnem, You should definitely try https://github.com/borisdj/EFCore.BulkExtensions. I wish https://github.com/dotnet/runtime/issues/28633 was released in EF Core 5 timeframe. – Maulik Modi May 28 '21 at 17:39
  • Re-creating a dbContext is really something to avoid. Keep in mind : don't do something just because it works... there is a proper way to deal with this problem as said by David Sherret. – David Aug 10 '22 at 16:40
  • @David The solution proposed by DavidSherret started to slow down after a few hours for me. Recreation is a proper way imo – Ogglas Aug 10 '22 at 18:57
  • @Ogglas, I'm quite curious to know why... I still think that recreating the dbcontext is not the solution or... It's a weird solution. – David Aug 16 '22 at 08:56
9

I just ran into this issue, and eventually stumbled upon a better solution for those using the typical .NET Core dependency injection. You can use a scoped DbContext for each operation. That will reset DbContext.ChangeTracker so that SaveChangesAsync() won't get bogged down checking entities from past iterations. Here is an example ASP.NET Core Controller method:

    /// <summary>
    /// An endpoint that processes a batch of records.
    /// </summary>
    /// <param name="provider">The service provider to create scoped DbContexts.
    /// This is injected by DI per the FromServices attribute.</param>
    /// <param name="records">The batch of records.</param>
    public async Task<IActionResult> PostRecords(
        [FromServices] IServiceProvider provider,
        Record[] records)
    {
        // The service scope factory is used to create a scope per iteration
        var serviceScopeFactory =
            provider.GetRequiredService<IServiceScopeFactory>();

        foreach (var record in records)
        {
            // At the end of the using block, scope.Dispose() will be called,
            // release the DbContext so it can be disposed/reset
            using (var scope = serviceScopeFactory.CreateScope())
            {
                var context = scope.ServiceProvider.GetService<MainDbContext>();

                // Query and modify database records as needed

                await context.SaveChangesAsync();
            }
        }

        return Ok();
    }

Given that ASP.NET Core projects typically use DbContextPool, this doesn't even create/destroy the DbContext objects. (In case you were interested, DbContextPool actually calls DbContext.ResetState() and DbContext.Resurrect(), but I wouldn't recommend calling those directly from your code, as they will probably change in future releases.) https://github.com/aspnet/EntityFrameworkCore/blob/v2.2.1/src/EFCore/Internal/DbContextPool.cs#L157

Matt
  • 659
  • 6
  • 11
2

From EF Core 3.0 there is an internal API that can reset the ChangeTracker. Do not use this in production code, I mention it as it may help someone in testing depending on the scenario.

((IResettableService)ChangeTracker).ResetState();

As the comment on the code says;

This is an internal API that supports the Entity Framework Core infrastructure and not subject to the same compatibility standards as public APIs. It may be changed or removed without notice in any release. You should only use it directly in your code with extreme caution and knowing that doing so can result in application failures when updating to a new Entity Framework Core release.

Stuart Hallows
  • 8,795
  • 5
  • 45
  • 57
  • Although a good find, it doesn't seem to actually clear the state, seems to just create a new one and then reset that one but leaves the original as-is. But implementation might have changed internally. Would be good to see how to access the StateManager and clear the entries there. – CularBytes Apr 04 '22 at 10:32