Error "Collection was modified; enumeration operation may not execute.", when adding from ConcurrentDictionary

Question

In my code, I initially cache existing objects of my entity DataEntry in form of a ConcurrentDictionary. From parallel tasks, I try to read pre-cached data entries. If none exists, I want to create a new one.

ConcurrentDictionary<string, DataEntry> dataEntryDict  = new ConcurrentDictionary<string, DataEntry>(
  await db.DataEntries
    .Where(de => allObjIDs.Contains(de.PAObjID))
    .IncludeOptimized(de => de.WorkSchedules)
    .ToDictionaryAsync(a => a.PAObjID, a => a)
);
var allDataEntryNumbers = new ConcurrentHashSet<DataEntryStruct>();

await Task.WhenAll(allDataEntryNumbers.Batch(20).Select(async workOrderBatch => {
  var gwoResp = await MyServiceCall();

  foreach (dsyWorkOrder01TtyWorkOrder currDetail in gwoResp.dsyWorkOrder01) {
    // Get or create element
    DataEntry currentEntry = dataEntryDict.GetOrAdd(
      currDetail.Obj,
      key => {
        var newDe = new DataEntry();
        db.DataEntries.Add(newDe); // This seems to be the line, where the exception is thrown
        return newDe;
      }
    );

    // Set regular fields
    currentEntry.ApplyTtyWorkOrder(currDetail, resourceDict);
  }
}

If I call that (simplified) code, I get the error message from the title. But not always. The method can be called from the UI, which works in 100 % of the cases, but it can also be called from a background worker, which triggers every night.

This resulted in the following error:

Full import failed with a/an InvalidOperationException. Collection was modified; enumeration operation may not execute.

at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource) at System.Collections.Generic.Dictionary2.ValueCollection.Enumerator.MoveNext() at System.Data.Entity.Core.Objects.ObjectStateManager.GetEntityEntriesForDetectChanges(Dictionary2 entityStore, List1& entries) at System.Data.Entity.Core.Objects.ObjectStateManager.GetEntityEntriesForDetectChanges() at System.Data.Entity.Core.Objects.ObjectStateManager.DetectChanges() at System.Data.Entity.Internal.InternalContext.DetectChanges(Boolean force) at System.Data.Entity.Internal.Linq.InternalSet1.ActOnSet(Action action, EntityState newState, Object entity, String methodName) at System.Data.Entity.Internal.Linq.InternalSet1.Add(Object entity) at System.Data.Entity.DbSet1.Add(TEntity entity) at Namespace.Import.ImportScheduler.<>c__DisplayClass9_1.b__16(String key) in C:\Users\Reichelt\source\repos\path\Import\FullImport.cs:line 295 at System.Collections.Concurrent.ConcurrentDictionary2.GetOrAdd(TKey key, Func2 valueFactory) at Namespace.Import.ImportScheduler.<>c__DisplayClass9_3.<b__5>d.MoveNext() in C:\Users\Reichelt\source\repos\path\Import\FullImport.cs:line 0 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Namespace.Import.ImportScheduler.d__9.MoveNext() in C:\Users\Reichelt\source\repos\path\Import\FullImport.cs:line 323

So you have any idea what's going wrong in this case?

If that is simplified, i would hate to maintain this code base. I think this question is unanswerable with out a minimal reproducible example . — TheGeneral, Mar 12 '20 at 08:23
Does this answer your question? [Collection was modified; enumeration operation may not execute](https://stackoverflow.com/questions/604831/collection-was-modified-enumeration-operation-may-not-execute) — Louis Go, Mar 12 '20 at 08:39
@MichaelRandall The actual code consists of more than 1000 lines and I cannot even reproduce the issue in my debug environment. Maybe somebody knows that EF has some issues with parallel adding and how to avoid it. Or do I need to lock the db.DataEntries list in some way? — André Reichelt, Mar 12 '20 at 09:03
@LouisGo I already read that thread before asking this question. My wild guess is, that another thread might access the DbSet at the same time and that leads to some collissions. I wonder, however, why this only happens when called from the background worker. And even then, it's a problem that only occurs sporadicly. — André Reichelt, Mar 12 '20 at 09:05
@AndréReichelt Did you try `gwoResp.dsyWorkOrder01.ToArray()` in the post? It's a workaround, but it may not suit your use case. foreach will throw this exception whenever it finds collection is changed. Or do you want to remove the root cause of it? — Louis Go, Mar 12 '20 at 09:17
You did not provide any detail about gwoResp. I'll assume `gwoResp` might modify its `dsyWorkOrder01`. If you can't reproduce it, change `dsyWorkOrder01` getter to create new a snapshot of array to check if problem persists. — Louis Go, Mar 12 '20 at 10:15
Copy your dsyWorkOrder01 before foreach, do not enumerate `dsyWorkOrder01` directly from gwoResp. To see of problem persist. — Louis Go, Mar 15 '20 at 02:14
_"you have any idea what's going wrong in this case?"_ -- yes. The collection is being modified while you try to enumerate it, just as the exception says, and as the marked duplicate explains. Based on your notes so far, you probably have a concurrency bug in your code. We can't help debug it unless you provide a good [mcve] that reliably reproduces the problem. — Peter Duniho, Mar 15 '20 at 03:34

Louis Go · Answer 1 · 2020-03-15T03:08:18.187

-1

According to Collection was modified; enumeration operation may not execute

It worths a try to use a snapshot collection in foreach because gwoResp.dsyWorkOrder01 might be changed during foreach operation.

From

foreach (dsyWorkOrder01TtyWorkOrder currDetail in gwoResp.dsyWorkOrder01) 
// following codes are ommited.

To

var arr = gwoResp.dsyWorkOrder01.ToArray();

foreach (dsyWorkOrder01TtyWorkOrder currDetail in arr ) 
// following codes are ommited.

edited Mar 15 '20 at 03:08

answered Mar 15 '20 at 03:03

Louis Go

2,213
2
16
29

I still strongly doubt that this is the error. There is nothing in my code that manipulates that array (it's not even a list). – André Reichelt Mar 16 '20 at 08:43
@AndréReichelt There is no more info from your post. That's why I post it. – Louis Go Mar 16 '20 at 08:51
I added a comment with additional info two days ago. – André Reichelt Mar 16 '20 at 08:52
Do you mean "The object is a result set of a WCF service."? That doesn't answer any detail. Or maybe a minimal reproducible example would help. – Louis Go Mar 16 '20 at 08:53
The problem is, that I can't even reproduce the error myself. It happens on the nightly import, but never when triggered manually. I even tried to modify my manually triggered method to use a `Timer` object as well, but that doesn't help either. – André Reichelt Mar 16 '20 at 09:16
@AndréReichelt Did you try the method of my answer? That exception is clearly stating your ConcurrentDictionary is changed. So make a snapshot (an array) should work. It's hard to find root cause in stackoverflow because the bug lies in a hugh codebase. – Louis Go Mar 16 '20 at 09:38
I've added a lock statement around the DbSet. We'll see tomorrow, if the error is gone then. Otherwise I will try your approach. Even after two hours of testing, I wasn't able to trigger the error on my machine. – André Reichelt Mar 16 '20 at 11:41
@AndréReichelt Lock might not help. The exception is detected at `MoveNext` which means a modification occurs between foreach item `i` to item `i + 1`. – Louis Go Mar 16 '20 at 14:46
I still don't understand, though, what would modify that array. I don't have any writing access to that object in my code. It's a WCF service response object where I just iterate over that array. It's also defined inside the `Task`, so side effects from other threads should also not be a problem here. – André Reichelt Mar 16 '20 at 16:25

Error "Collection was modified; enumeration operation may not execute.", when adding from ConcurrentDictionary

1 Answers1