Caching and lazy loading with entity framework

Question

let's say I have an application, for example a web site, where my objectcontext leaves during the time of a request. Some datas I load with EF should be cached to avoid to read in DB and improve performance.

Ok, I read my datas with EF, I put my object in cache (says AppFabric, not in memory cache), but related datas that can be lazy loaded are now null (and access to this property results in a nullreferenceexception). I don't want to load everything in one request, because it's going to be too long, so I want to keep the loading on demand and as soon as it's read, I would like to complete the cache with the new fetched datas.

Note :

• only read operations, no create/update/delete.
• Don't want to use second level cache like "EF Provider Wrappers" made by Jarek Kowalski

How can I do that ?

EDIT : I've built this samples with northwind database, it's working :

class Program
{
    static void Main(string[] args)
    {
        // normal use
        List<Products> allProductCached = null;
        using (NORTHWNDEntities1 db = new NORTHWNDEntities1())
        {
            allProductCached = db.Products.ToList().Clone<DbSet<Products>>();
            foreach (var product in db.Products.Where(e => e.UnitPrice > 100))
            {
                Console.WriteLine(product.ProductName + " => " + product.Suppliers.CompanyName);
            }
        }

        // try to use cache, but missing Suppliers
        using (NORTHWNDEntities1 db = new NORTHWNDEntities1())
        {
            foreach (var product in allProductCached.Where(e => e.UnitPrice > 100))
            {
                if (product.Suppliers == null)
                    product.Suppliers = db.Suppliers.FirstOrDefault(s => s.SupplierID == product.SupplierID).Clone<Suppliers>();
                Console.WriteLine(product.ProductName + " => " + product.Suppliers.CompanyName);
            }
        }

        // try to use full cache
        using (NORTHWNDEntities1 db = new NORTHWNDEntities1())
        {
            foreach (var product in allProductCached.Where(e => e.UnitPrice > 100))
            {
                Console.WriteLine(product.ProductName + " => " + product.Suppliers.CompanyName);
            }
        }
    }
}

public static class Ext
{
    public static List<Products> Clone<T>(this List<Products> list)
    {
        return list.Select(obj =>
            new Products
            {
                ProductName = obj.ProductName,
                SupplierID = obj.SupplierID,
                UnitPrice = obj.UnitPrice
            }).ToList();
    }

    public static Suppliers Clone<T>(this Suppliers obj)
    {
        if (obj == null)
            return null;
        return new Suppliers
        {
            SupplierID = obj.SupplierID,
            CompanyName = obj.CompanyName
        };
    }
}

The problem is that I have to copy everything (without missing a property) and test everywhere if the property is null and load the needed property. My code is of course more and more complex, so that will be a problem if I miss something. No other solution ?

Answer 1

You cannot access the database in EF without an ObjectContext or a DbContext.

You can still use caching effectively, even if you don't have the original context any more.

Maybe your scenario is something like this... Imagine that you have some reference data that you use frequently. You do not want to hit the database each time you need it, so you store it in a cache. You also have per-user data that you don't want to cache. You have navigation properties from your user data to your reference data. You want to load your user data from the database, and have EF automatically "fix up" the navigation properties to point to the reference data.

For a request:

1. Create a new DbContext.
2. Retrieve reference data from the cache.
3. Make a deep copy of the reference objects. (You probably don't want to have the same entities attached to multiple contexts simultaneously.)
4. Attach each of the reference objects to the context. (e.g. with DbSet.Attach())
5. Execute whatever queries are required to load the per-user data. EF will automatically "fix up" the references to the reference data.
6. Identify newly loaded entities that could be cached. Ensure that they contain no references to entities that should not be cached, then save them to the cache.
7. Dispose of the context.

Cloned Objects and Lazy Loading

Lazy loading in EF is usually accomplished using dynamic proxies. The idea is that you make all properties that could potentially be loaded dynamically virtual. Whenever EF creates an instance of your entity type, it actually substitutes a derived type instead, and that derived type has the lazy loading logic in its overridden version of your properties.

This is all well and good, but in this scenario you are attaching entity objects to the context that were not created by EF. You created them, using a method called Clone. You instantiated the real POCO entity, not some mysterious EF dynamic proxy type. That means you won't get lazy loading on these entities.

The solution is simple. The Clone method must take an additional argument: the DbContext. Don't use the entity's constructor to create a new instance. Instead, use DbSet.Create(). This will return a dynamic proxy. Then initialize its properties to create a clone of the reference entity. Then attach it to the context.

Here is the code you might use to clone a single Products entity:

public static Products Clone(this Products product, DbContext context)
{
    var set = context.Set<Products>();
    var clone = set.Create();
    clone.ProductName = product.ProductName;
    clone.SupplierID = product.SupplierID;
    clone.UnitProce = product.UnitPrice;

    // Initialize collection so you don't have to do the null check, but
    // if the property is virtual and proxy creation is enabled, it should get lazy loaded.
    clone.Suppliers = new List<Suppliers>();

    return clone;
}

Code Sample

namespace EFCacheLazyLoadDemo
{
    using System;
    using System.Collections.Generic;
    using System.ComponentModel.DataAnnotations.Schema;
    using System.Data.Entity;
    using System.Linq;

    class Program
    {
        static void Main(string[] args)
        {
            // Add some demo data.
            using (MyContext c = new MyContext())
            {
                var sampleData = new Master 
                { 
                    Details = 
                    { 
                        new Detail { SomeDetail = "Cod" },
                        new Detail { SomeDetail = "Haddock" },
                        new Detail { SomeDetail = "Perch" }
                    } 
                };

                c.Masters.Add(sampleData);
                c.SaveChanges();
            }

            Master cachedMaster;

            using (MyContext c = new MyContext())
            {
                c.Configuration.LazyLoadingEnabled = false;
                c.Configuration.ProxyCreationEnabled = false;

                // We don't load the details here.  And we don't even need a proxy either.
                cachedMaster = c.Masters.First();
            }

            Console.WriteLine("Reference entity details count: {0}.", cachedMaster.Details.Count);

            using (MyContext c = new MyContext())
            {
                var liveMaster = cachedMaster.DeepCopy(c);

                c.Masters.Attach(liveMaster);

                Console.WriteLine("Re-attached entity details count: {0}.", liveMaster.Details.Count);
            }

            Console.ReadKey();
        }
    }

    public static class MasterExtensions
    {
        public static Master DeepCopy(this Master source, MyContext context)
        {
            var copy = context.Masters.Create();
            copy.MasterId = source.MasterId;

            foreach (var d in source.Details)
            {
                var copyDetail = context.Details.Create();
                copyDetail.DetailId = d.DetailId;
                copyDetail.MasterId = d.MasterId;
                copyDetail.Master = copy;
                copyDetail.SomeDetail = d.SomeDetail;
            }

            return copy;
        }
    }

    public class MyContext : DbContext
    {
        static MyContext()
        {
            // Just for demo purposes, re-create db each time this runs.
            Database.SetInitializer(new DropCreateDatabaseAlways<MyContext>());
        }

        public DbSet<Master> Masters { get { return this.Set<Master>(); } }

        public DbSet<Detail> Details { get { return this.Set<Detail>(); } }
    }

    public class Master
    {
        public Master()
        {
            this.Details = new List<Detail>();
        }

        public int MasterId { get; set; }

        public virtual List<Detail> Details { get; private set; }
    }

    public class Detail
    {
        public int DetailId { get; set; }

        public string SomeDetail { get; set; }

        public int MasterId { get; set; }

        [ForeignKey("MasterId")]
        public Master Master { get; set; }
    }
}

Here is a sample model, different from yours, that shows how to get this working in principle.