Entity Framework duplicates lookup values when assigned to nested list member

Question

This solution involves three entities, Client, Competency and WeaponType.

• A Client instance can have zero-or-more Competency instances within a List<Competency> member.
• A Competency instance can have one-or-more WeaponType instances within a List<WeaponType> member. (WeaponType is our lookup member)

Before updating the DbContext, a new List object is assigned to the client. This represents the full updated list of competencies for the client, where old competencies might have been removed and new ones created.

The problem experienced is that dbContext.SaveChanges() causes duplicate WeaponType entries to be created.

Here is the code for my entities:

public class Client : Person
    {
        public ICollection<CompetencyCertificate> CompetencyCertificates
        {
            get;
            set;
        }

    }

public class CompetencyCertificate
    {


        public Int64 Id { get; set; }

        [Required]
        public string CertificateNumber { get; set; }

        [Required]
        public List<WeaponType> CompetencyTypes { get; set; }    

    }



 public class WeaponType
    {
        public Int16 Id { get; set; }

        [Required]
        public string Name { get; set; }

    }

And herewith the code for saving my updated client and competency info (which reflects my attempts to overcome this problem as well:

 private void SaveClientProfile()
        {
            HttpRequestBase rb = this.Request;

            string sId = "";
            if (rb.Form["Id"] != null)
                sId = rb.Form["Id"];
            Int64 int64_id = 0;
            if (sId.Trim().Length > 0)
                int64_id = Int64.Parse(sId);
            Client client = loadOrCreateClient(int64_id);

            //Set the newly submitted form data for the client

            client.IDSocialSecurityPassNum = rb.Form["IDNumber"];
            client.EmailAddress = rb.Form["EmailAddress"];
            client.NickName = rb.Form["Name"];
            client.Surname = rb.Form["Surname"];

            //MAP AND TRANSLATE JSON COLLECTION OBJECTS TO ENTITY COLLECTIONS, UPDATE THE CONTEXT    

            Mapper.CreateMap<Client_Competency_ViewModel, CompetencyCertificate>();
            client.CompetencyCertificates = Mapper.Map<List<CompetencyCertificate>>(System.Web.Helpers.Json.Decode<System.Collections.Generic.List<Client_Competency_ViewModel>>(rb.Form["CompetencyCollection"]));    

            //PREVENT EF FROM DUPLICATING LOOKUP VALUES
            AttachLookup<WeaponType>(JCGunsDb.WeaponTypes.ToList<WeaponType>());


            //FNIALISE AND SAVE
            dbContext.UserId = User.Identity.GetUserName();
            dbContext.SaveChanges();
        }



private void AttachLookup<T>(ICollection<T> itemsToAttach) where T : class
        {
            foreach(T item in itemsToAttach)
            {
                JCGunsDb.Entry(item).State = EntityState.Unchanged;
            }
        }

I can confirm the JSON parsing and mapping in the above code works as expected - the Id's for existing entities are in tact and new entity Id's are set to 0.

What am I doing that is causing this behaviour? How do I fix it?

---

UPDATE: As recommended by Gert, I have tried to implement a solution utilising GraphDiff (which seems to be an exact fit for my requirements). However, I am struggling to get it to work. Here is what I have done (as per Github issue raised):

I have the following:

Client Client >> List CompetencyCertificates CompetencyCertificate >> List CompetencyTypes

I load a client object from the database, and then assign new List values to the List members mentioned above.

Subsequently, I call the following code:

dbContext.UpdateGraph(client, map => map
                .OwnedCollection(cc => cc.CompetencyCertificates, with => with
                    .AssociatedCollection(kt => kt.CompetencyTypes))
                );

dbContext.SaveChanges();

Here is the stacktrace for the exception that gets thrown on the UpdateGraph invocation:

Member 'CurrentValues' cannot be called for the entity of type 'CompetencyCertificate' because the entity does not exist in the context. To add an entity to the context call the Add or Attach method of DbSet.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: Member 'CurrentValues' cannot be called for the entity of type 'CompetencyCertificate' because the entity does not exist in the context. To add an entity to the context call the Add or Attach method of DbSet.

Source Error:

Line 138: Line 139: //UPDATE GRAPH OF DETACHED ENTITIES Line 140: dbContext.UpdateGraph(client, map => map Line 141: .OwnedCollection(cc => cc.CompetencyCertificates, with => with Line 142: .AssociatedCollection(kt => kt.CompetencyTypes))

Source File: [Not Important] Line: 140

Stack Trace:

[InvalidOperationException: Member 'CurrentValues' cannot be called for the entity of type 'CompetencyCertificate' because the entity does not exist in the context. To add an entity to the context call the Add or Attach method of DbSet.]
System.Data.Entity.Internal.InternalEntityEntry.ValidateNotDetachedAndInitializeRelatedEnd(String method) +102
System.Data.Entity.Internal.InternalEntityEntry.ValidateStateToGetValues(String method, EntityState invalidState) +55
System.Data.Entity.Internal.InternalEntityEntry.get_CurrentValues() +53 System.Data.Entity.Infrastructure.DbEntityEntry.get_CurrentValues() +44 RefactorThis.GraphDiff.DbContextExtensions.RecursiveGraphUpdate(DbContext context, Object dataStoreEntity, Object updatingEntity, UpdateMember member) +942
RefactorThis.GraphDiff.DbContextExtensions.UpdateGraph(DbContext context, T entity, Expression1 mapping) +631
JCGunsOnline.Controllers.ClientController.SaveClientProfile() in c:\Users\Ben\Dropbox\Mighty IT\Active Projects\JCGunsOnline\JCGunsOnline\Views\Client\ClientController.cs:140 JCGunsOnline.Controllers.ClientController.SubmitStep1() in c:\Users\Ben\Dropbox\Mighty IT\Active Projects\JCGunsOnline\JCGunsOnline\Views\Client\ClientController.cs:60 lambda_method(Closure , ControllerBase , Object[] ) +101
System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +59
System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary2 parameters) +435
System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary2 parameters) +60
System.Web.Mvc.Async.ActionInvocation.InvokeSynchronousActionMethod() +76 System.Web.Mvc.Async.AsyncControllerActionInvoker.<BeginInvokeSynchronousActionMethod>b__39(IAsyncResult asyncResult, ActionInvocation innerInvokeState) +36
System.Web.Mvc.Async.WrappedAsyncResult2.CallEndDelegate(IAsyncResult asyncResult) +73
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +136
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +102
System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult) +49
System.Web.Mvc.Async.AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f() +117 System.Web.Mvc.Async.<>c__DisplayClass48.<InvokeActionMethodFilterAsynchronouslyRecursive>b__41() +323 System.Web.Mvc.Async.<>c__DisplayClass33.<BeginInvokeActionMethodWithFilters>b__32(IAsyncResult asyncResult) +44
System.Web.Mvc.Async.WrappedAsyncResult1.CallEndDelegate(IAsyncResult asyncResult) +47
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +136
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +102
System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult) +50
System.Web.Mvc.Async.<>c__DisplayClass2b.<BeginInvokeAction>b__1c() +72 System.Web.Mvc.Async.<>c__DisplayClass21.<BeginInvokeAction>b__1e(IAsyncResult asyncResult) +185
System.Web.Mvc.Async.WrappedAsyncResult1.CallEndDelegate(IAsyncResult asyncResult) +42
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +133
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +56
System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult) +40
System.Web.Mvc.Controller.<BeginExecuteCore>b__1d(IAsyncResult asyncResult, ExecuteCoreState innerState) +34
System.Web.Mvc.Async.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) +70
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +139
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +59
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +40
System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult) +44 System.Web.Mvc.Controller.<BeginExecute>b__15(IAsyncResult asyncResult, Controller controller) +39
System.Web.Mvc.Async.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) +62
System.Web.Mvc.Async.WrappedAsyncResultBase1.End() +139
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +59
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +40 System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult) +39
System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.EndExecute(IAsyncResult asyncResult) +39
System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__5(IAsyncResult asyncResult, ProcessRequestState innerState) +39
System.Web.Mvc.Async.WrappedAsyncVoid1.CallEndDelegate(IAsyncResult asyncResult) +70
System.Web.Mvc.Async.WrappedAsyncResultBase`1.End() +139
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +59
System.Web.Mvc.Async.AsyncResultWrapper.End(IAsyncResult asyncResult, Object tag) +40
System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) +40 System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result) +38
System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +9514928 System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +155

Answer 1

So I managed to solve the problem and as such am summarising it here for future reference.

From the code listed in my original question, I was de-serialising from JSON to Enities, thereby basically creating a disconnected graph (because the graph was not loaded from the database and therefore no tracking occurred on the entities.

Entity Framework 6 (and earlier) does not support working with disconnected graphs. (See https://entityframework.codeplex.com/workitem/864)

As @Gert Arnold mentioned above, there is a plug-in component called GraphDiff which does support it. (You can download it from https://github.com/refactorthis/GraphDiff).

I strongly suggest that you build the code from source and do not make use of the Nuget package, as it was out-of-date when I used it and subsequently ran in a battery of bugs which were already fixed in the latest version.

Lastly, keep in mind that GraphDiff does not yet support working with connected graphs / tracked entities, therefore you have to call the .AsNoTracking() method when loading the data for your disconnected graph.

Answer 2

The problem is in the line

client.CompetencyCertificates = Mapper.Map<....

All CompetencyCertificates in the collection start as unattached objects when they are deserialized. When you assign the deserialized collection to CompetencyCertificates, all CompetencyCertificate objects change from Detached to Added.

This state change is one that causes all Detached objects in an object graph to be marked as Added as well. So at this point, all WeaponTypes are Added and will be saved as new objects if you don't do anything about it.

If you know for sure that all WeaponType objects will always be existing objects, I think the quickest fix would be to loop through all new CompetencyCertificate objects and mark their WeaponTypes as Unchanged.

This is probably what you're trying to do in AttachLookup, but it seems to me that an entirely different context is involved there, so dbContext's change tracker never gets involved in this.