Entity Framework Core 2.1 failling to update entities with relations

Question

I’m currently having issues with EF core 2.1 and a web api used by a native client to update an object which contains several levels of embedded objects. I’ve already read theses two topics:

Entity Framework Core: Fail to update Entity with nested value objects

https://learn.microsoft.com/en-us/ef/core/saving/disconnected-entities

I’ve learned through this that it is indeed not that obvious for now to update objects in EF Core 2. But I’ve not yet managed to find a solution that works. On each attempt I’m having an exception telling me that a “step” is already tracked by EF.

My model looks like this:

//CIApplication the root class I’m trying to update
public class CIApplication : ConfigurationItem // -> derive of BaseEntity which holds the ID and some other properties  
{

    //Collection of DeploymentScenario
    public virtual ICollection<DeploymentScenario> DeploymentScenarios { get; set; }

    //Collection of SoftwareMeteringRules
    public virtual ICollection<SoftwareMeteringRule> SoftwareMeteringRules { get; set; }
}

//Deployment Scenario which have a one to many relationship with Application. A deployment scenario contain two lists of steps

public class DeploymentScenario : BaseEntity
{

    //Collection of substeps
    public virtual ICollection<Step> InstallSteps { get; set; }
    public virtual ICollection<Step> UninstallSteps { get; set; }

    //Navigation properties Parent CI
    public Guid? ParentCIID { get; set; }
    public virtual CIApplication ParentCI { get; set; }
}

//Step, which is also quite complex and is also self-referencing

public class Step : BaseEntity
{

    public string ScriptBlock { get; set; }


    //Parent Step Navigation property
    public Guid? ParentStepID { get; set; }
    public virtual Step ParentStep { get; set; }

    //Parent InstallDeploymentScenario Navigation property
    public Guid? ParentInstallDeploymentScenarioID { get; set; }
    public virtual DeploymentScenario ParentInstallDeploymentScenario { get; set; }

    //Parent InstallDeploymentScenario Navigation property
    public Guid? ParentUninstallDeploymentScenarioID { get; set; }
    public virtual DeploymentScenario ParentUninstallDeploymentScenario { get; set; }

    //Collection of sub steps
    public virtual ICollection<Step> SubSteps { get; set; }

    //Collection of input variables
    public virtual List<ScriptVariable> InputVariables { get; set; }
    //Collection of output variables
    public virtual List<ScriptVariable> OutPutVariables { get; set; }

}

Here’s my update method, I know it’s ugly and it shouldn’t be in the controller but I’m changing it every two hours as I try to implement solutions if find on the web. So this is the last iteration coming from https://learn.microsoft.com/en-us/ef/core/saving/disconnected-entities

public async Task<IActionResult> PutCIApplication([FromRoute] Guid id, [FromBody] CIApplication cIApplication)
    {
        _logger.LogWarning("Updating CIApplication " + cIApplication.Name);

        if (!ModelState.IsValid)
        {
            return BadRequest(ModelState);
        }

        if (id != cIApplication.ID)
        {
            return BadRequest();
        }

        var cIApplicationInDB = _context.CIApplications
            .Include(c => c.Translations)
            .Include(c => c.DeploymentScenarios).ThenInclude(d => d.InstallSteps).ThenInclude(s => s.SubSteps)
            .Include(c => c.DeploymentScenarios).ThenInclude(d => d.UninstallSteps).ThenInclude(s => s.SubSteps)
            .Include(c => c.SoftwareMeteringRules)
            .Include(c => c.Catalogs)
            .Include(c => c.Categories)
            .Include(c => c.OwnerCompany)
            .SingleOrDefault(c => c.ID == id);

        _context.Entry(cIApplicationInDB).CurrentValues.SetValues(cIApplication);

        foreach(var ds in cIApplication.DeploymentScenarios)
        {
            var existingDeploymentScenario = cIApplicationInDB.DeploymentScenarios.FirstOrDefault(d => d.ID == ds.ID);

            if (existingDeploymentScenario == null)
            {
                cIApplicationInDB.DeploymentScenarios.Add(ds);
            }
            else
            {
                _context.Entry(existingDeploymentScenario).CurrentValues.SetValues(ds);

                foreach(var step in existingDeploymentScenario.InstallSteps)
                {
                    var existingStep = existingDeploymentScenario.InstallSteps.FirstOrDefault(s => s.ID == step.ID);

                    if (existingStep == null)
                    {
                        existingDeploymentScenario.InstallSteps.Add(step);
                    }
                    else
                    {
                        _context.Entry(existingStep).CurrentValues.SetValues(step);
                    }
                }
            }
        }
        foreach(var ds in cIApplicationInDB.DeploymentScenarios)
        {
            if(!cIApplication.DeploymentScenarios.Any(d => d.ID == ds.ID))
            {
                _context.Remove(ds);
            }
        }

        //_context.Update(cIApplication);
        try
        {
            await _context.SaveChangesAsync();
        }
        catch (DbUpdateConcurrencyException e)
        {
            if (!CIApplicationExists(id))
            {
                return NotFound();
            }
            else
            {
                throw;
            }
        }
        catch(Exception e)
        {
        }

        return Ok(cIApplication);
    }

So far I’m getting this exception : The instance of entity type 'Step' cannot be tracked because another instance with the key value '{ID: e29b3c1c-2e06-4c7b-b0cd-f8f1c5ccb7b6}' is already being tracked.

I paid attention that no “get” operation was made previously by the client and even if it was the case I’ve put AsNoTracking on my get methods. The only operation made before the update by the client is “ _context.CIApplications.Any(e => e.ID == id);” to ckeck if I should Add a new record or update an existing one.

I’ve been fighting with this issue since few days so I would really appreciate if someone could help me getting in the right direction. Many thanks

UPDATE :

I added the following code in my controller :

var existingStep = existingDeploymentScenario.InstallSteps.FirstOrDefault(s => s.ID == step.ID);
                    entries = _context.ChangeTracker.Entries();
                    if (existingStep == null)
                    {
                        existingDeploymentScenario.InstallSteps.Add(step);
                        entries = _context.ChangeTracker.Entries();
                    }

The entries = _context.ChangeTracker.Entries(); line raise the "step is already tracked" exception right after adding the new deploymentScenario which contains the also new step.

Just before it the new deploymentScenario and step are not in the tracker and I've check in DB their IDs are not duplicated.

I also check my Post method and now it's failing too... I reverted it to the default methods with no fancy stuff Inside :

[HttpPost]
    public async Task<IActionResult> PostCIApplication([FromBody] CIApplication cIApplication)
    {
        if (!ModelState.IsValid)
        {
            return BadRequest(ModelState);
        }
        var entries = _context.ChangeTracker.Entries();
        _context.CIApplications.Add(cIApplication);
        entries = _context.ChangeTracker.Entries();
        await _context.SaveChangesAsync();
        entries = _context.ChangeTracker.Entries();
        return CreatedAtAction("GetCIApplication", new { id = cIApplication.ID }, cIApplication);
    }

Entries are empty at the beginning and the _context.CIApplications.Add(cIApplication); line is still raising the exception still about the only one step included in the deploymentscenario...

So there obviously somthing wrong when I try to add stuff in my context, but right now I'm feeling totally lost. It may can help here how I declare my context in startup :

services.AddDbContext<MyAppContext>(options =>
            options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection"),
            b => b.MigrationsAssembly("DeployFactoryDataModel")),
            ServiceLifetime.Transient
            );

Add my context class :

public class MyAppContext : DbContext
{
    private readonly IHttpContextAccessor _contextAccessor;
    public MyAppContext(DbContextOptions<MyAppContext> options, IHttpContextAccessor contextAccessor) : base(options)
    {
        _contextAccessor = contextAccessor;
    }


    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {

        optionsBuilder.EnableSensitiveDataLogging();
    }

    public DbSet<Step> Steps { get; set; }
    //public DbSet<Sequence> Sequences { get; set; }
    public DbSet<DeploymentScenario> DeploymentScenarios { get; set; }
    public DbSet<ConfigurationItem> ConfigurationItems { get; set; }
    public DbSet<CIApplication> CIApplications { get; set; }
    public DbSet<SoftwareMeteringRule> SoftwareMeteringRules { get; set; }
    public DbSet<Category> Categories { get; set; }
    public DbSet<ConfigurationItemCategory> ConfigurationItemsCategories { get; set; }
    public DbSet<Company> Companies { get; set; }
    public DbSet<User> Users { get; set; }
    public DbSet<Group> Groups { get; set; }
    public DbSet<Catalog> Catalogs { get; set; }
    public DbSet<CIDriver> CIDrivers { get; set; }
    public DbSet<DriverCompatiblityEntry> DriverCompatiblityEntries { get; set; }
    public DbSet<ScriptVariable> ScriptVariables { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        //Step one to many with step for sub steps
        modelBuilder.Entity<Step>().HasMany(s => s.SubSteps).WithOne(s => s.ParentStep).HasForeignKey(s => s.ParentStepID);

        //Step one to many with step for variables
        modelBuilder.Entity<Step>().HasMany(s => s.InputVariables).WithOne(s => s.ParentInputStep).HasForeignKey(s => s.ParentInputStepID);
        modelBuilder.Entity<Step>().HasMany(s => s.OutPutVariables).WithOne(s => s.ParentOutputStep).HasForeignKey(s => s.ParentOutputStepID);

        //Step one to many with sequence
        //modelBuilder.Entity<Step>().HasOne(step => step.ParentSequence).WithMany(seq => seq.Steps).HasForeignKey(step => step.ParentSequenceID).OnDelete(DeleteBehavior.Cascade);

        //DeploymentScenario One to many with install steps
        modelBuilder.Entity<DeploymentScenario>().HasMany(d => d.InstallSteps).WithOne(s => s.ParentInstallDeploymentScenario).HasForeignKey(s => s.ParentInstallDeploymentScenarioID);

        //DeploymentScenario One to many with uninstall steps
        modelBuilder.Entity<DeploymentScenario>().HasMany(d => d.UninstallSteps).WithOne(s => s.ParentUninstallDeploymentScenario).HasForeignKey(s => s.ParentUninstallDeploymentScenarioID);

        //DeploymentScenario one to one with sequences
        //modelBuilder.Entity<DeploymentScenario>().HasOne(ds => ds.InstallSequence).WithOne(seq => seq.IDeploymentScenario).HasForeignKey<DeploymentScenario>(ds => ds.InstallSequenceID).OnDelete(DeleteBehavior.Cascade);
        //modelBuilder.Entity<DeploymentScenario>().HasOne(ds => ds.UninstallSequence).WithOne(seq => seq.UDeploymentScenario).HasForeignKey<DeploymentScenario>(ds => ds.UninstallSequenceID);

        //Step MUI config
        modelBuilder.Entity<Step>().Ignore(s => s.Description);
        modelBuilder.Entity<Step>().HasMany(s => s.Translations).WithOne().HasForeignKey(x => x.StepTranslationId);

        //Sequence MUI config
        //modelBuilder.Entity<Sequence>().Ignore(s => s.Description);
        //modelBuilder.Entity<Sequence>().HasMany(s => s.Translations).WithOne().HasForeignKey(x => x.SequenceTranslationId);

        //DeploymentScenario MUI config
        modelBuilder.Entity<DeploymentScenario>().Ignore(s => s.Name);
        modelBuilder.Entity<DeploymentScenario>().Ignore(s => s.Description);
        modelBuilder.Entity<DeploymentScenario>().HasMany(s => s.Translations).WithOne().HasForeignKey(x => x.DeploymentScenarioTranslationId);

        //CIApplication  relations
        //CIApplication one to many relation with Deployment Scenario
        modelBuilder.Entity<CIApplication>().HasMany(ci => ci.DeploymentScenarios).WithOne(d => d.ParentCI).HasForeignKey(d => d.ParentCIID).OnDelete(DeleteBehavior.Cascade);
        modelBuilder.Entity<CIApplication>().HasMany(ci => ci.SoftwareMeteringRules).WithOne(d => d.ParentCI).HasForeignKey(d => d.ParentCIID).OnDelete(DeleteBehavior.Cascade);

        // CIDriver relations
        // CIAPpplication one to many relation with DriverCompatibilityEntry
        modelBuilder.Entity<CIDriver>().HasMany(ci => ci.CompatibilityList).WithOne(c => c.ParentCI).HasForeignKey(c => c.ParentCIID).OnDelete(DeleteBehavior.Restrict);

        //ConfigurationItem MUI config
        modelBuilder.Entity<ConfigurationItem>().Ignore(s => s.Name);
        modelBuilder.Entity<ConfigurationItem>().Ignore(s => s.Description);
        modelBuilder.Entity<ConfigurationItem>().HasMany(s => s.Translations).WithOne().HasForeignKey(x => x.ConfigurationItemTranslationId);

        //category MUI config
        modelBuilder.Entity<Category>().Ignore(s => s.Name);
        modelBuilder.Entity<Category>().Ignore(s => s.Description);
        modelBuilder.Entity<Category>().HasMany(s => s.Translations).WithOne().HasForeignKey(x => x.CategoryTranslationId);

        //CI Categories Many to Many
        modelBuilder.Entity<ConfigurationItemCategory>().HasKey(cc => new { cc.CategoryId, cc.CIId });
        modelBuilder.Entity<ConfigurationItemCategory>().HasOne(cc => cc.Category).WithMany(cat => cat.ConfigurationItems).HasForeignKey(cc => cc.CategoryId);
        modelBuilder.Entity<ConfigurationItemCategory>().HasOne(cc => cc.ConfigurationItem).WithMany(ci => ci.Categories).HasForeignKey(cc => cc.CIId);

        //CI Catalog Many to Many
        modelBuilder.Entity<CICatalog>().HasKey(cc => new { cc.CatalogId, cc.ConfigurationItemId });
        modelBuilder.Entity<CICatalog>().HasOne(cc => cc.Catalog).WithMany(cat => cat.CIs).HasForeignKey(cc => cc.CatalogId);
        modelBuilder.Entity<CICatalog>().HasOne(cc => cc.ConfigurationItem).WithMany(ci => ci.Catalogs).HasForeignKey(cc => cc.ConfigurationItemId);

        //Company Customers Many to Many
        modelBuilder.Entity<CompanyCustomers>().HasKey(cc => new { cc.CustomerId, cc.ProviderId });
        modelBuilder.Entity<CompanyCustomers>().HasOne(cc => cc.Provider).WithMany(p => p.Customers).HasForeignKey(cc => cc.ProviderId).OnDelete(DeleteBehavior.Restrict);
        modelBuilder.Entity<CompanyCustomers>().HasOne(cc => cc.Customer).WithMany(c => c.Providers).HasForeignKey(cc => cc.CustomerId);

        //Company Catalog Many to Many
        modelBuilder.Entity<CompanyCatalog>().HasKey(cc => new { cc.CatalogId, cc.CompanyId });
        modelBuilder.Entity<CompanyCatalog>().HasOne(cc => cc.Catalog).WithMany(c => c.Companies).HasForeignKey(cc => cc.CatalogId);
        modelBuilder.Entity<CompanyCatalog>().HasOne(cc => cc.Company).WithMany(c => c.Catalogs).HasForeignKey(cc => cc.CompanyId);

        //Author Catalog Many to Many
        modelBuilder.Entity<CatalogAuthors>().HasKey(ca => new { ca.AuthorId, ca.CatalogId });
        modelBuilder.Entity<CatalogAuthors>().HasOne(ca => ca.Catalog).WithMany(c => c.Authors).HasForeignKey(ca => ca.CatalogId);
        modelBuilder.Entity<CatalogAuthors>().HasOne(ca => ca.Author).WithMany(a => a.AuthoringCatalogs).HasForeignKey(ca => ca.AuthorId);

        //Company one to many with owned Catalog
        modelBuilder.Entity<Company>().HasMany(c => c.OwnedCatalogs).WithOne(c => c.OwnerCompany).HasForeignKey(c => c.OwnerCompanyID).OnDelete(DeleteBehavior.Restrict);
        //Company one to many with owned Categories
        modelBuilder.Entity<Company>().HasMany(c => c.OwnedCategories).WithOne(c => c.OwnerCompany).HasForeignKey(c => c.OwnerCompanyID).OnDelete(DeleteBehavior.Restrict);
        //Company one to many with owned CIs
        modelBuilder.Entity<Company>().HasMany(c => c.OwnedCIs).WithOne(c => c.OwnerCompany).HasForeignKey(c => c.OwnerCompanyID).OnDelete(DeleteBehavior.Restrict);

        //CIDriver one to many with DriverCompatibilityEntry
        modelBuilder.Entity<CIDriver>().HasMany(c => c.CompatibilityList).WithOne(c => c.ParentCI).HasForeignKey(c => c.ParentCIID).OnDelete(DeleteBehavior.Restrict);

        //User Group Many to Many
        modelBuilder.Entity<UserGroup>().HasKey(ug => new { ug.UserId, ug.GroupId });
        modelBuilder.Entity<UserGroup>().HasOne(cg => cg.User).WithMany(ci => ci.Groups).HasForeignKey(cg => cg.UserId);
        modelBuilder.Entity<UserGroup>().HasOne(cg => cg.Group).WithMany(ci => ci.Users).HasForeignKey(cg => cg.GroupId);

        //User one to many with Company
        modelBuilder.Entity<Company>().HasMany(c => c.Employees).WithOne(u => u.Employer).HasForeignKey(u => u.EmployerID).OnDelete(DeleteBehavior.Restrict);
    }

UPDATE 2

Here's a one drive link to a minima repro example. I haven't implemented PUT in the client as the post method already reproduce the issue.

https://1drv.ms/u/s!AsO87EeN0Fnsk7dDRY3CJeeLT-4Vag

Answer 1

You are enumerating over existing steps here, and search for existing step in existing steps collection which does not make sense.

 foreach(var step in existingDeploymentScenario.InstallSteps)
     var existingStep = existingDeploymentScenario.InstallSteps
         .FirstOrDefault(s => s.ID == step.ID);

while it should probably be:

foreach(var step in ds.InstallSteps)

Answer 2

I figured it out and I feel quite ashamed.

thanks to all of you I finally suspected that the client and the ay it handle the data was responsible of the issue.

Turns out that when the client creates a deployment scenario, it creates a step and assign it both to the installStep and uninstallSteps lists thus causing the issue...

I was so sure the uninstallstep list was not used I didn't even lokked at it when debugging.