IQueryable.Include(string path) not modifying query to include joins

Question

Using Entity Framework 5, I have a generic method that retrieves entities from my context with optional parameters to filter, include related entities, and set the order of results. When I pass the method a set of include properties, however, it never modifies the query with joins to include the related entities. Any ideas why my query isn't updating?

Method:

public virtual IQueryable<TEntity> Get(
    Expression<Func<TEntity, bool>> filter = null,
    string includeProperties = "",
    Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null)
{        
    IQueryable<TEntity> query = dbSet.AsExpandable();

    if (filter != null)
    {
        query = query.Where(filter);
    }

    foreach (var includeProperty in includeProperties.Split
        (new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
    {
        query = query.Include(includeProperty);
    }

    if (orderBy != null)
    {
        return orderBy(query);
    }
    else
    {
        return query;
    }
}

Query before foreach (var includeProperty... that remains unchanged afterwards

{SELECT 
[Extent1].[Pid] AS [Pid],     
[Extent1].[Created] AS [Created], 
[Extent1].[Creator] AS [Creator]
FROM [Administrator] AS [Extent1]}

EDIT...More Info

Originally I was using the following method call: AdministratorDTO admin = unitOfWork.AdministratorComponent.GetByID(userPid); on the following POCO class:

public virtual TEntity GetByID(object id)
{
    return dbSet.Find(id);
}

public class Administrator : IPrincipal
{
    [Key]
    [Required]
    [StringLength(1024)]
    [Display(Name = "PID")]
    public string Pid { get; set; }

    public DateTime Created { get; set; }

    public string Creator { get; set; }

    ...

    public ICollection<Role> Roles { get; set; }

    public ICollection<Area> Areas { get; set; }

    ...
}

And using AutoMapper to map to a DTO with the following configuration:

public class AdministratorDTO
{
    [Key]
    [Required]
    [StringLength(1024)]
    [Display(Name = "PID")]
    public string Pid { get; set; }

    ...

    public string[] Roles { get; set; }

    public string[] Areas { get; set; }
}


public class WebApiApplication : System.Web.HttpApplication
{
...
AutoMapperConfiguration.Configure();
}

public static class AutoMapperConfiguration
{
    public static void Configure()
    {
        ConfigureAdministratorMapping();
        ...
    }

    private static void ConfigureAdministratorMapping()
    {
        Mapper.CreateMap<Administrator, AdministratorDTO>()
            .ForMember(dest => dest.Roles, 
                opt => opt.MapFrom(src => src.Roles == null ? 
                    null : src.Roles.Select(r => r.Name).ToArray())) 
            .ForMember(dest => dest.Areas, 
                opt => opt.MapFrom(src => src.Areas == null ?
                    null : src.Areas.Select(a => a.Id).ToArray()));

        ...
    }
}

public class BusinessComponent<TEntity, TDto> 
    : IBusinessComponent<TEntity, TDto>
        where TEntity : class
        where TDto : class
{
    ...
    protected TDto Flatten(TEntity entity)
    {
        return Mapper.Map<TEntity, TDto>(entity);
    }
}

My understanding was that if I didn't mark the Administrator's navigation properties (Areas and Roles) as virtual they would be eagerly loaded, but I kept getting an empty string[] in my DTO.

I looked at the TEntity parameter going into my Flatten method and Areas/Roles were null before I called Map, so I don't think it was something to do with AutoMapper.

Next I tried using the following method call:

AdministratorDTO admin = unitOfWork.AdministratorComponent
    .Get(filter: a => a.Pid == "csherman", includeProperties: "Roles, Areas")
    .SingleOrDefault();

Finally, just in case the Include was being ignored because the navigation properties were not virtual, I added the virtual keyword to both Areas and Roles on my Administrator class. When I did this, both the GetByID and the Get(filter: ..., includeProperties: ...) methods worked, thereby including the Areas/Roles in my TEntity Flatten parameter and populating the string arrays in my DTO.

Problem solved I suppose, but...

Question is, especially for the GetById method, why did it work with the virtual keyword but not without?

If EF actually factors in the projection from the time of the original method call, why would these entities be included?

Answer 1

Includes don't work if you do any sort of projection after the include.

Here's a post that deals with this and how to work around it.

And here's an SO question that deals with it as well.

EDIT: In response to your edit I tried looking around to see if I could find a reason why the virtual keyword would make your Include work but not work without it. I couldn't find anything in passing that would directly answer that.

Here's what I think is happening, though: The virtual keyword, on navigation properties, tells Entity Framework that this property should employ lazy loading. If there are times that you want to load them eagerly is when you would use Include. I think that that's what Include is built for. I think it tries to look specifically for virtual properties with that name, and when it can't find it, dies gracefully without exception. If you don't mark it as virtual I think the implementation of Include misses it completely. I base this guess on the fact that none of the articles that I've seen on Include mentioned it outside the context of lazy loading--which would mean using the virtual keyword.

Answer 2

There's a serious problem with IQueryable.Include: the underlying object must be of type ObjectQuery or DbQuery, otherwise this method will not work!

Include is leaky abstraction and it works only with Entity framework. EF 4.1 already contains Include over generic IQueryable but it internally only converts passed generic IQueryable to generic ObjectQuery or DbQuery and calls their Include.

https://stackoverflow.com/a/6791874/2444725

When I pass the method a set of include properties, however, it never modifies the query with joins to include the related entities. Any ideas why my query isn't updating?

You seem to be using LinqKit:

IQueryable<TEntity> query = dbSet.AsExpandable();

Extension method AsExpandable gets IQueryable parameter and returns a new object of type ExpandableQuery, that decorates the original ObjectQuery or DbQuery object. Extension method IQueryable.Include, applied to ExpandableQuery, can't make the conversion and silently skips.

why did it work with the virtual keyword but not without?

Virtual method has nothing to do with include. Instead it turns on lazy loading. It means that navigation properties are not loaded on the first request, but are loaded on separate requests when these properties are actually used.