19

So, I've read all the Q&A's here on SO regarding the subject of whether or not to expose IQueryable to the rest of your project or not (see here, and here), and I've ultimately decided that I don't want to expose IQueryable to anything but my Model. Because IQueryable is tied to certain persistence implementations I don't like the idea of locking myself into this. Similarly, I'm not sure how good I feel about classes further down the call chain modifying the actual query that aren't in the repository.

So, does anyone have any suggestions for how to write a clean and concise Repository without doing this? One problem I see, is my Repository will blow up from a ton of methods for various things I need to filter my query off of.

Having a bunch of:

IEnumerable GetProductsSinceDate(DateTime date);  
IEnumberable GetProductsByName(string name);  
IEnumberable GetProductsByID(int ID);

If I was allowing IQueryable to be passed around I could easily have a generic repository that looked like:

public interface IRepository<T> where T : class
{
    T GetById(int id);
    IQueryable<T> GetAll();
    void InsertOnSubmit(T entity);
    void DeleteOnSubmit(T entity);
    void SubmitChanges();
}

However, if you aren't using IQueryable then methods like GetAll() aren't really practical since lazy evaluation won't be taking place down the line. I don't want to return 10,000 records only to use 10 of them later.

What is the answer here? In Conery's MVC Storefront he created another layer called the "Service" layer which received IQueryable results from the respository and was responsible for applying various filters.

Is this what I should do, or something similar? Have my repository return IQueryable but restrict access to it by hiding it behind a bunch of filter classes like GetProductByName, which will return a concrete type like IList or IEnumerable?

Community
  • 1
  • 1
mmcdole
  • 91,488
  • 60
  • 186
  • 222
  • I'm in the same shoes. Don't want to expose IQueryable as i definitely see it leaking outside of the repository and being misused. I also don't want to be bound to linq-capable persistence only. Something better might come along and switching won't be possible at that point. Personally, i see 2 options here a) bloated repositories b) DAO over Repository. I am actually considering choosing b as my option. Although it goes against DDD principle, i'd rather have granularity than huge and complicated repositories. –  Jan 13 '10 at 07:06
  • Ultimately, the best solution likely involves figuring out a way to create a query as a parameter without having the set we're querying available yet. I'm working on trying to sort out how to do this, but I'm not quite there yet. i.e. setting up something where you can define a query like this: var groupFilter = new FilterBuilder.Where(g => g.groupId == GroupEnum.ExampleGroup); and then later be able to call repository.GetGroups(groupFilter). Ultimately you build up all the filters you want, and pass them into the repo, and the repo applies them all and returns a List instead of IQueryable. – Mir Dec 30 '11 at 17:38

5 Answers5

5

Exposing an IQueryable is a very viable solution and this is how most of the Repository implementations out there doing right now. (Including SharpArchitecture and FubuMVC contrib, as well.)

This is where you are wrong:

However, if you aren't using IQueryable then methods like GetAll() aren't really practical since lazy evaluation won't be taking place down the line. I don't want to return 10,000 records only to use 10 of them later.

This is not realy true. Your example is correct and you should rename GetAll() to a more informative name.

It DOESN'T return all of the items if you call it. That is what IQueryable is for. The concept is called "deferred loading", as it only loads the data (and makes database requests) when you enumerate the IQueryable.

So, let's say I have a method like this:

IQueryable<T> Retrieve() { ... }

Then, I can call it like this:

Repository.Retrieve<Customer>().Single(c => c.ID == myID);

This ONLY retrieves one row from the database.

And this:

Repository.Retrieve<Customer>().Where(c => c.FirstName == "Joe").OrderBy(c => c.LastName);

This also generates a corresponding query and is only executed when you enumerate it. (It generates an expression tree from the query, and then the query provider should translate that into an appropriate query against the data source.)

You can read more about it in this MSDN article.

Venemo
  • 18,515
  • 13
  • 84
  • 125
  • 5
    I think that's what he understands too. What he is saying, is that when _you aren't_ using `IQueryable` but instead using `IEnumerable` for the GetAll, then you will get all even if you just use 10. – Rodi May 30 '11 at 08:08
  • @Ergwun and &Rodi - apparently you don't understand how `IQueryable` works. I'm sorry for that. – Venemo Nov 14 '14 at 09:43
  • 1
    @Venemo I'm not disagreeing with your explanation of `IQueryable`. The point is the question is asking about *not* using `IQueryable`, but `IEnumerable` instead. – Ergwun Nov 16 '14 at 01:18
3

Rob's method really doesn't solve your core problem, and that is not wanting to write individual methods for each type of query you'd like to run, and unfortunately if you're not using IQueryable then that is what you are left with.

Sure the methods might be in the "service" layer, but it still means having to write "GetProductsByName, GetProductsByDate"...

The other method is something like:

GetProducts(QueryObject);

This might give you some benefit over using IQueryable in that you can constrain what is returned.

ChadT
  • 7,598
  • 2
  • 40
  • 57
3

hmm.. I solved this in many ways depending on the type of ORM i use.
The main idea is to have one repository base class and one query method that takes so many parameters indicating all possible where/orderby/expand|include/paging/etc options.

Here is a quick and dirty sample using LINQ to NHibernate (of course the entire repository should be implementation detail):

public class RepositoryBase
    {
        private ISession Session;

        public RepositoryBase()
        {
            Session = SessionPlaceHolder.Session;
        }



        public TEntity[] GetPaged<TEntity>(IEnumerable<Expression<Func<TEntity, bool>>> filters,
            IEnumerable<Expression<Func<TEntity, object>>> relatedObjects,
            IEnumerable<Expression<Func<TEntity, object>>> orderCriterias,
            IEnumerable<Expression<Func<TEntity, object>>> descOrderCriterias,
            int pageNumber, int pageSize, out int totalPages)
        {
            INHibernateQueryable<TEntity> nhQuery = Session.Linq<TEntity>();

            if (relatedObjects != null)
                foreach (var relatedObject in relatedObjects)
                {
                    if (relatedObject == null) continue;
                    nhQuery = nhQuery.Expand(relatedObject);
                }

            IQueryable<TEntity> query = nhQuery;

            if (filters != null)
                foreach (var filter in filters)
                {
                    if (filter == null) continue;
                    query = query.Where(filter);
                }

            bool pagingEnabled = pageSize > 0;

            if (pagingEnabled)
                totalPages = (int) Math.Ceiling((decimal) query.Count()/(decimal) pageSize);
            else
                totalPages = 1;

            if (orderCriterias != null)
                foreach (var orderCriteria in orderCriterias)
                {
                    if (orderCriteria == null) continue;
                    query = query.OrderBy(orderCriteria);
                }

            if (descOrderCriterias != null) 
                foreach (var descOrderCriteria in descOrderCriterias)
                {
                    if (descOrderCriteria == null) continue;
                    query = query.OrderByDescending(descOrderCriteria);
                }

            if (pagingEnabled)
                query = query.Skip(pageSize*(pageNumber - 1)).Take(pageSize);

            return query.ToArray();
        }
    }

Normally you'll want to add many chaining overloads as shortcuts when you don't need paging for example, etc..

Here is another dirty one. Sorry I'm not sure if I can expose the final ones. Those were drafts and are OK to show:

using Context = Project.Services.Repositories.EntityFrameworkContext;
using EntitiesContext = Project.Domain.DomainSpecificEntitiesContext;    
namespace Project.Services.Repositories
{
    public class EntityFrameworkRepository : IRepository
    {
        #region IRepository Members

        public bool TryFindOne<T>(Expression<Func<T, bool>> filter, out T result)
        {
            result = Find(filter, null).FirstOrDefault();

            return !Equals(result, default(T));
        }

        public T FindOne<T>(Expression<Func<T, bool>> filter)
        {
            T result;
            if (TryFindOne(filter, out result))
                return result;

            return default(T);
        }

        public IList<T> Find<T>() where T : class, IEntityWithKey
        {
            int count;
            return new List<T>(Find<T>(null, null, 0, 0, out count));
        }

        public IList<T> Find<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort)
        {
            int count;
            return new List<T>(Find(filter, sort, 0, 0, out count));
        }

        public IEnumerable<T> Find<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort, int pageSize,
                                      int pageNumber, out int count)
        {
            return ExecuteQuery(filter, sort, pageSize, pageNumber, out count) ?? new T[] {};
        }

        public bool Save<T>(T entity)
        {
            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            EntityKey key = context.CreateEntityKey(GetEntitySetName(entity.GetType()), entity);

            object originalItem;
            if (context.TryGetObjectByKey(key, out originalItem))
            {
                context.ApplyPropertyChanges(key.EntitySetName, entity);
            }
            else
            {
                context.AddObject(GetEntitySetName(entity.GetType()), entity);
                //Attach(context, entity);
            }

            return context.SaveChanges() > 0;
        }

        public bool Delete<T>(Expression<Func<T, bool>> filter)
        {
            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            int numberOfObjectsFound = 0;
            foreach (T entity in context.CreateQuery<T>(GetEntitySetName(typeof (T))).Where(filter))
            {
                context.DeleteObject(entity);
                ++numberOfObjectsFound;
            }

            return context.SaveChanges() >= numberOfObjectsFound;
        }

        #endregion

        protected IEnumerable<T> ExecuteQuery<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort,
                                                 int pageSize, int pageNumber,
                                                 out int count)
        {
            IEnumerable<T> result;

            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            ObjectQuery<T> originalQuery = CreateQuery<T>(context);
            IQueryable<T> query = originalQuery;

            if (filter != null)
                query = query.Where(filter);

            if (sort != null)
                query = query.OrderBy(sort);

            if (pageSize > 0)
            {
                int pageIndex = pageNumber > 0 ? pageNumber - 1 : 0;
                query = query.Skip(pageIndex).Take(pageSize);

                count = query.Count();
            }
            else 
                count = -1;


            result = ExecuteQuery(context, query);

            //if no paging total count is count of the entire result set
            if (count == -1) count = result.Count();

            return result;
        }

        protected internal event Action<ObjectContext, IEnumerable> EntitiesFound;

        protected void OnEntitiesFound<T>(ObjectContext context, params T[] entities)
        {
            if (EntitiesFound != null && entities != null && entities.Length > 0)
            {
                EntitiesFound(context, entities);
            }
        }

        //Allowing room for system-specific-requirement extensibility
        protected Action<IEnumerable> ItemsFound;

        protected IEnumerable<T> ExecuteQuery<T>(ObjectContext context, IQueryable<T> query)
        {
            IEnumerable<T> result = null;

            if (query is ObjectQuery)
            {
                var objectQuery = (ObjectQuery<T>) query;

                objectQuery.EnablePlanCaching = false;
                objectQuery.MergeOption = MergeOption.PreserveChanges;

                result = new List<T>(objectQuery);

                if (ItemsFound != null)
                    ItemsFound(result);

                return result;
            }

            return result;
        }

        internal static RelationshipManager GetRelationshipManager(object entity)
        {
            var entityWithRelationships = entity as IEntityWithRelationships;
            if (entityWithRelationships != null)
            {
                return entityWithRelationships.RelationshipManager;
            }

            return null;
        }


        protected ObjectQuery<T> CreateQuery<T>(ObjectContext context)
        {
            ObjectQuery<T> query = context.CreateQuery<T>(GetEntitySetName(typeof (T)));
            query = this.AggregateEntities(query);
            return query;
        }

        protected virtual ObjectQuery<T> AggregateEntities<T>(ObjectQuery<T> query)
        {
            return query;
        }

        private static string GetEntitySetName(Type entityType)
        {
            return string.Format("{0}Set", entityType.Name);
        }
    }

    public class EntityFrameworkContext
    {
        private const string CtxKey = "ctx";

        private bool contextInitialized
        {
            get { return HttpContext.Current.Items[CtxKey] != null;  }
        }

        public EntitiesContext Context
        {
            get
            {
                if (contextInitialized == false)
                {
                    HttpContext.Current.Items[CtxKey] = new EntitiesContext(ConfigurationManager.ConnectionStrings["CoonectionStringName"].ToString());
                }

                return (EntitiesContext)HttpContext.Current.Items[CtxKey];
            }
        }

        public void TrulyDispose()
        {
            if (contextInitialized)
            {
                Context.Dispose();
                HttpContext.Current.Items[CtxKey] = null;
            }
        }
    }

    internal static class EntityFrameworkExtensions
    {
        internal static ObjectQuery<T> Include<T>(this ObjectQuery<T> query,
                                                  Expression<Func<T, object>> propertyToInclude)
        {
            string include = string.Join(".", propertyToInclude.Body.ToString().Split('.').Skip(1).ToArray());

            const string collectionsLinqProxy = ".First()";
            include = include.Replace(collectionsLinqProxy, "");

            return query.Include(include);
        }

        internal static string After(this string original, string search)
        {
            if (string.IsNullOrEmpty(original))
                return string.Empty;

            int index = original.IndexOf(search);
            return original.Substring(index + search.Length);
        }
    }
}

In Conery's MVC Storefront he created another layer called the "Service" layer which received IQueryable results from the respository and was responsible for applying various filters.

In all cases nobody should be interacting with the repository directly except the services layer.

Most flexible thing is to let Services interact with Repository whatever way they want, same as in above code (yet through one single point -as in example also- to write DRY code and find a place for optimization).
However, the more right way in terms of common DDD patterns is to use the "Specification" pattern, where you encapsulate all your filters, etc in Variables (Class Members, in LINQ typically of delegate types). LINQ can take big optimization benefit out of this when you combine it with "Compiled queries". If you google the {Specification Pattern} and {LINQ Compiled Queries} you'll get closer to what I mean here.

Meligy
  • 35,654
  • 11
  • 85
  • 109
  • Here I use IEnumerable so that it's easy to write method calls using array initializer and it's possible (if composing elements dynamically) to use generic List<> or so. Maybe sometime you want also to create shortcut that takes single where clause and single ordrby, et.. – Meligy Nov 17 '09 at 11:12
  • I was going to blog a cleaner implementation of this repository. Maybe this post makes it sooner :). Thanks for asking this BTW! – Meligy Nov 17 '09 at 12:11
  • So the bottom line of this lengthy example is: "It's OK to expose IQueryable through Repositories as long as access to repositories is limited to Service classes. Is that right? – Ahmed Nov 19 '09 at 11:16
  • The bottom line is: There is no single best approach. You can expose your IQueryable to services, or better expose IQueryable functionality (via lambda expressions for filtering, etc..) so that you have one place to improve queries performance through. You can take this even further and make the lambdas AND eager/lazy loading options be members in specification classes as well. – Meligy Nov 19 '09 at 11:49
  • One important issue here is that if `IQueryable` is exposed to other layers it may fail. But seems to me that `public TEntity[] GetPaged()` will fail too if the filters are not safe. Eg. `date => (DateTime.Now - date).TotalDays > 1)`. Am I right? – roozbeh S Nov 11 '18 at 22:27
0

Having struggled to find a viable solution to this problem myself, there's what seems to be a good solution in the Implementing the Repository and Unit of Work Patterns in an ASP.NET MVC Application (9 of 10) article.

public virtual IEnumerable<TEntity> Get(
    Expression<Func<TEntity, bool>> filter = null,
    Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
    string includeProperties = "") 
    {
        IQueryable<TEntity> query = dbSet;

        if (filter != null)
        {
            query = query.Where(filter);
        }

        foreach (var includeProperty in includeProperties.Split
            (new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
        {
            query = query.Include(includeProperty);
        }

        if (orderBy != null)
        {
            return orderBy(query).ToList();
        }
        else
        {
            return query.ToList();
        }
    }

The article doesn't talk about this exact issue but does talk about generic, reusable repository methods.

So far this is all I've been able to come up with as a solution.

Joseph Woodward
  • 9,191
  • 5
  • 44
  • 63
0

I ended up creating two sets of methods, ones that return IEnumerable (in your case IQueryable), and ones that return Collection (pull the content before sending it out of the repository.)

This allows me to do both build ad hoc queries in Services outside the repository as well as use Repository methods directly returning side effect resistant Collections. In other words, joining two Repository Entities together results in one select query, instead of one select query for each entity found.

I imagine you could set your protection level to keep truly bad things from happening.

Zachary Scott
  • 20,968
  • 35
  • 123
  • 205
  • 1
    `IEnumerable` is in memory collection and so very different from IQueryable. An IEnumerable will retrieve everything directly into memory and thus filtering, joining etc. will be in memory with all data. In other words: having `Collection` next to `IEnumerable` will not have significant value for this purpose. – Rodi May 30 '11 at 08:14