virtual keyword, Include extension method, lazy loading, eager loading - how does loading related objects actually work

Question

Loading related object in MVC can be pretty confusing.

There are lots of terms you need to be aware of and learn if you really want to know what you're doing when writing your entity model classes and controllers.

A couple of questions that I've had for a very long time are: How does the virtual keyword work and when should I use it? And how does the Include extension method work and when should I use it?

Here's what I'm talking about;

virtual keyword:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace LazyLoading.Models
{
    public class Brand
    {
        public int BrandId { get; set; }
        public string Name { get; set; }
        public int ManufacturerId { get; set; }
        public virtual Manufacturer Manufacturer { get; set; }
    }
}

And the Include extension method:

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.Entity;
using System.Linq;
using System.Web;
using System.Web.Mvc;
using LazyLoading.Models;

namespace LazyLoading.Controllers
{
    public class LazyLoadingStoreController : Controller
    {
        private UsersContext db = new UsersContext();

        //
        // GET: /LazyLoadingStore/

        public ActionResult Index()
        {
            var brands = db.Brands.Include(b => b.Manufacturer);
            return View(brands.ToList());
        }

        ... Other action methods ...

Notice that the Index() action method is autogenerated by Visual Studio. Yes, Visual Studio automatically added the .Include(b => b.Manufacturer). That's pretty nice.

Answer 1

Note: It took me too long to write this answer to just discard it when two other appeared ...

Virtual keyword works together with two properties DbContext.Configuration:

• ProxyCreationEnabled - allows EF crating dynamic proxy when the object is created by the EF
• LazyLoadingEnabled - allows dynamic proxy to load related entities when the navigation property is used for the first time

Lazy loading is transparently implemented through dynamic proxy. Dynamic proxy is class derived from your entity which is created and compiled by EF at runtime. It overrides your virtual navigation properties and implements logic checking if related entities were already loaded or not. If not it triggers a loading on the context (issues a new query to the database). Lazy loading can be executed only in the scope of the context to which the entity is attached - if you dispose the context you cannot use it.

Dynamic proxy creation is controlled by the first mentioned property. Once the entity instance is created as proxied you cannot "remove" the proxy. Also if you create entity without proxy (for example by calling constructor yourselves) you cannot add it later (but you can use DbSet<T>.Create instead of constructor to get the proxied instance).

The second property can be changed during live time of your entity instances so you can avoid unnecessary queries to the database when working with your entities by changing it to false (it is sometimes very useful).

Include represents eager loading. Eager loading loads related entities together with the main entity and it is executed as part of the main entity's query (it adds SQL joins to the query and builds a big result sets).

Benefit of eager loading is to get all data upfront with one roundtrip to the database. Especially if you know that you will need all of them it can be a way to go. Disadvantage of eager loading are very big result sets if you use too many includes as well as some limitations (you cannot add ordering or filtering to loaded entities - it always loads all related objects).

Benefit of lazy loading is to load data only when you really need them. It is helpful if you don't know upfront if you will really need them. Disadvantages are additional queries generated by EF in some scenarios when you don't expect them (any first access to the property will trigger the lazy loading - even Count on navigation collection will trigger loading of all data to be able to do count in your application instead of querying count from the database - this is called extra lazy loading and it is not natively supported by EF yet). Another big issue is N+1 problem. If you load several brands and you will access their manufacturer property by looping through all loaded brands without using eager loading, you will generate N+1 queries to database (where N is number of brands) - one for loading all brands and one for a manufacturer of each brand.

There is another option called explicit loading. It is like lazy loading but it is not transparently executed for you. You must execute it yourselves by using context class:

context.Entry(brand).Reference(b => b.Manufacturer).Load();

It is not very useful in this case but it would be useful if you have Brands navigation property on the Manufacturer class because you can do this:

var dataQuery = context.Entry(manufacturer).Collection(m => m.Brands).Query();

Now you have a IQueryable<Brand> instance and you can add any condition, ordering or even additional eager loading and execute it against the database.

Answer 2

I created a test MVC4 internet application.

Here's what I found:

First, create your entity model classes - notice the virtual keyword for the Manufacturer property:

public class Manufacturer
{
    public int ManufacturerId { get; set; }
    public string Name { get; set; }
    public ICollection<Brand> Brands { get; set; }
}

public class Brand
{
    public int BrandId { get; set; }
    public string Name { get; set; }
    public int ManufacturerId { get; set; }
    public virtual Manufacturer Manufacturer { get; set; }
}

Next, create your controller - I created (autogenerated using create new controller dialog) mine with CRUD action methods and views. Notice the Include extension method that was autogenerated automatically by Visual Studio thanks to the relationship in you Brand model class.

public class LazyLoadingStoreController : Controller
{
    private UsersContext db = new UsersContext();

    //
    // GET: /LazyLoadingStore/

    public ActionResult Index()
    {
        var brands = db.Brands.Include(b => b.Manufacturer);
        return View(brands.ToList());
    }

Let's remove the Include part for now so that our action method looks like this:

public ActionResult Index()
{
    var brands = db.Brands;
    return View(brands.ToList());
}

And this is how the Index view will look in Page Inspector after adding a couple of Brand objects - notice that Visual Studio automatically added the dropdown for Manufacturer and how it automatically scaffolded the Name column for Manufacturer - sweet!:

The Create action method:

//
// GET: /LazyLoadingStore/Create

public ActionResult Create()
{
    ViewBag.ManufacturerId = new SelectList(db.Manufacturers, "ManufacturerId", "Name");
    return View();
}

Awesome. Everything was autogenerated for us!

Now, what happens if we remove the virtual keyword from our Manufacturer property?

public class Brand
{
    public int BrandId { get; set; }
    public string Name { get; set; }
    public int ManufacturerId { get; set; }
    public Manufacturer Manufacturer { get; set; }
}

This is what will happen - our Manufacturer data is gone:

Okay, makes sense. What if I add back the Include extension method (with virtual still removed from the Manufacturer property)?

public ActionResult Index()
{
    var brands = db.Brands.Include(b => b.Manufacturer);
    return View(brands.ToList());
}

This is the result from adding back the Include extension method - The Manufacturer data is back!:

So that's how all that stuff work.

Next thing would be to explain what kind of T-SQL that gets generated behind the scenes in both cases (Lazy loading and Eager loading). That I'll leave to someone else. :)

Note: Visual Studio automatically generates Include(b => b.Manufacturer) wheather you add the virtual keyword or not.

Note2: Oh, yeah. Almost forgot. Here are some links to some good Microsoft resources.

• http://msdn.microsoft.com/en-us/data/jj574232.aspx
• http://msdn.microsoft.com/en-us/library/vstudio/bb896272(v=vs.100).aspx

The second link talks about performance considerations which the other link lacks if that is something that gets you going.

Answer 3

Lazy loading

Brand is a POCO (plain old CLR object). It is persistence ignorant. In other words: it does not know it is created by an Entity Framework data layer. It knows even less how to load its Manufacturer.

Still, when you do this

var brand = db.Brands.Find(1);
var manufacturer = brand.Manufacturer;

the Manufacturer is loaded on the fly ("lazily"). If you monitor the SQL that's sent to the database you'll see that a second query is emitted to fetch the Manufacturer.

That is because under the hood, EF does not create a Brand instance, but a derived type, a proxy, that's stuffed with wiring to execute lazy loading. That's why the virtual modifier is required for lazy loading to be enabled: the proxy must be able to override it.

Lazy loading is typically used in smart client applications where a context has a relatively long life span (e.g. context per form). Although I must say that even in smart client applications using short-lived contexts is beneficial and perfectly possible.

Eager loading

Eager loading means that you load an object with adhering objects (parents and/or children) in one take. That's what the Include method is for. In the example

db.Brands.Include(b => b.Manufacturer)

you'll see that EF creates a SQL query with a join and that accessing a Brand's Manufacturer does not spawn a separate query any more.

Eager loading is the way to go in most situations (esp. in disconnected scenarios), because the recommended way to deal with context instances is to use them and dispose them for each unit of work. Thus, lazy loading is not an option, because for a navigation property to be lazy loaded the context must be alive.