How can you update a Linq Expression with additional parameters?

Question

I have a Linq Expression, which may be altered depending on certain conditions. An example of what I would like to do (left blank the bit I am not sure about):

Expression<Func<Project, bool>> filter = (Project p) => p.UserName == "Bob";
if(showArchived)
{
    // update filter to add && p.Archived
}
// query the database when the filter is built
IEnumerable<Project> projects = unitOfWork.ProjectRepository.Get(filter);

How do I update the filter to add any extra parameters?

At the moment all the records are retrieved, then I use a Where to further filter the results. However that results in more queries to the database than are strictly necessary.

IEnumerable<Project> projects = unitOfWork.ProjectRepository.Get(filter);
if(showArchived)
{
    projects = projects.Where(p => p.Archived);
}

Get method is using the GenericRepository pattern:

public class GenericRepository<TEntity> where TEntity : class
{
    internal ProgrammeDBContext context;
    internal DbSet<TEntity> dbSet;

    public GenericRepository(ProgrammeDBContext context)
    {
        this.context = context;
        this.dbSet = context.Set<TEntity>();
    }

    public virtual IEnumerable<TEntity> Get(
        Expression<Func<TEntity, bool>> filter = null,
        Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
        string includeProperties = "")
    {
        IQueryable<TEntity> query = dbSet;

        if (filter != null)
        {
            query = query.Where(filter);
        }

        foreach (var includeProperty in includeProperties.Split
            (new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
        {
            query = query.Include(includeProperty);
        }

        if (orderBy != null)
        {
            return orderBy(query).ToList();
        }
        else
        {
            return query.ToList();
        }
    }

    public virtual TEntity GetByID(object id)
    {
        return dbSet.Find(id);
    }

    public virtual void Insert(TEntity entity)
    {
        dbSet.Add(entity);
    }

    public virtual void Delete(object id)
    {
        TEntity entityToDelete = dbSet.Find(id);
        Delete(entityToDelete);
    }

    public virtual void Delete(TEntity entityToDelete)
    {
        if (context.Entry(entityToDelete).State == EntityState.Detached)
        {
            dbSet.Attach(entityToDelete);
        }
        dbSet.Remove(entityToDelete);
    }

    public virtual void Update(TEntity entityToUpdate)
    {
        dbSet.Attach(entityToUpdate);
        context.Entry(entityToUpdate).State = EntityState.Modified;
    }

    public virtual IEnumerable<TEntity> GetWithRawSql(string query, params object[] parameters)
    {
        return dbSet.SqlQuery(query, parameters).ToList();
    }
}

Update
Created some extension methods based on the code below by Marc Gravell and David B, solves the problem for me

public static class LinqExtensionMethods
{
    public static Expression<Func<T, bool>> CombineOr<T>(params Expression<Func<T, bool>>[] filters)
    {
        return filters.CombineOr();
    }

    public static Expression<Func<T, bool>> CombineOr<T>(this IEnumerable<Expression<Func<T, bool>>> filters)
    {
        if (!filters.Any())
        {
            Expression<Func<T, bool>> alwaysTrue = x => true;
            return alwaysTrue;
        }
        Expression<Func<T, bool>> firstFilter = filters.First();

        var lastFilter = firstFilter;
        Expression<Func<T, bool>> result = null;
        foreach (var nextFilter in filters.Skip(1))
        {
            var nextExpression = new ReplaceVisitor(lastFilter.Parameters[0], nextFilter.Parameters[0]).Visit(lastFilter.Body);
            result = Expression.Lambda<Func<T, bool>>(Expression.OrElse(nextExpression, nextFilter.Body), nextFilter.Parameters);
            lastFilter = nextFilter;
        }
        return result;
    }

    public static Expression<Func<T, bool>> CombineAnd<T>(params Expression<Func<T, bool>>[] filters)
    {
        return filters.CombineAnd();
    }

    public static Expression<Func<T, bool>> CombineAnd<T>(this IEnumerable<Expression<Func<T, bool>>> filters)
    {
        if (!filters.Any())
        {
            Expression<Func<T, bool>> alwaysTrue = x => true;
            return alwaysTrue;
        }
        Expression<Func<T, bool>> firstFilter = filters.First();

        var lastFilter = firstFilter;
        Expression<Func<T, bool>> result = null;
        foreach (var nextFilter in filters.Skip(1))
        {
            var nextExpression = new ReplaceVisitor(lastFilter.Parameters[0], nextFilter.Parameters[0]).Visit(lastFilter.Body);
            result = Expression.Lambda<Func<T, bool>>(Expression.AndAlso(nextExpression, nextFilter.Body), nextFilter.Parameters);
            lastFilter = nextFilter;
        }
        return result;
    }

    class ReplaceVisitor : ExpressionVisitor
    {
        private readonly Expression from, to;
        public ReplaceVisitor(Expression from, Expression to)
        {
            this.from = from;
            this.to = to;
        }
        public override Expression Visit(Expression node)
        {
            return node == from ? to : base.Visit(node);
        }
    }
}

What is the return type and interlans of `ProjectRepository.Get(filter);`? — Oybek, Mar 13 '12 at 11:57
What is `showAchieved`? Does it enumerates `projects` variable? — Oybek, Mar 13 '12 at 11:58
Now that you've added Get, the ToList() is a pain; I've added an expression-rewriting example that you should be able to use to combine the two filters **before** calling Get — Marc Gravell, Mar 13 '12 at 12:21

Marc Gravell · Accepted Answer · 2012-03-13T13:22:48.213

If I understand the question, then most likely here's the problem:

IEnumerable<Project> projects = unitOfWork.ProjectRepository.Get(filter);

Any work on projects is going to be using Enumerable, not Queryable; it should probably be:

IQueryable<Project> projects = unitOfWork.ProjectRepository.Get(filter);
if(showArchived)
{
    projects = projects.Where(p => p.Archived);
}

The latter is composable, and .Where should work as you expect, building up a more restrictive query before sending it to the server.

Your other option is to rewrite the filter to combine before sending:

using System;
using System.Linq.Expressions;

static class Program
{
    static void Main()
    {
        Expression<Func<Foo, bool>> filter1 = x => x.A > 1;
        Expression<Func<Foo, bool>> filter2 = x => x.B > 2.5;

        // combine two predicates:
        // need to rewrite one of the lambdas, swapping in the parameter from the other
        var rewrittenBody1 = new ReplaceVisitor(
            filter1.Parameters[0], filter2.Parameters[0]).Visit(filter1.Body);
        var newFilter = Expression.Lambda<Func<Foo, bool>>(
            Expression.AndAlso(rewrittenBody1, filter2.Body), filter2.Parameters);
        // newFilter is equivalent to: x => x.A > 1 && x.B > 2.5
    }
}
class Foo
{
    public int A { get; set; }
    public float B { get; set; }
}
class ReplaceVisitor : ExpressionVisitor
{
    private readonly Expression from, to;
    public ReplaceVisitor(Expression from, Expression to)
    {
        this.from = from;
        this.to = to;
    }
    public override Expression Visit(Expression node)
    {
        return node == from ? to : base.Visit(node);
    }
}

Or re-written in a way to allow convenient usage:

using System;
using System.Linq.Expressions;

static class Program
{
    static void Main()
    {
        Expression<Func<Foo, bool>> filter = x => x.A > 1;

        bool applySecondFilter = true;
        if(applySecondFilter)
        {
            filter = Combine(filter, x => x.B > 2.5);
        }
        var data = repo.Get(filter);
    }
    static Expression<Func<T,bool>> Combine<T>(Expression<Func<T,bool>> filter1, Expression<Func<T,bool>> filter2)
    {
        // combine two predicates:
        // need to rewrite one of the lambdas, swapping in the parameter from the other
        var rewrittenBody1 = new ReplaceVisitor(
            filter1.Parameters[0], filter2.Parameters[0]).Visit(filter1.Body);
        var newFilter = Expression.Lambda<Func<T, bool>>(
            Expression.AndAlso(rewrittenBody1, filter2.Body), filter2.Parameters);
        return newFilter;
    }
}
class Foo
{
    public int A { get; set; }
    public float B { get; set; }
}
class ReplaceVisitor : ExpressionVisitor
{
    private readonly Expression from, to;
    public ReplaceVisitor(Expression from, Expression to)
    {
        this.from = from;
        this.to = to;
    }
    public override Expression Visit(Expression node)
    {
        return node == from ? to : base.Visit(node);
    }
}

@Oybek the difference between `Enumerable.Where` and `Queryable.Where` matters **very much** - can you clarify what you are saying doesn't matter? — Marc Gravell, Mar 13 '12 at 11:57
It works, it is just that the ProjectRepository.Get(filter) gets all the records for the database, then the Where hits the database again. I want to do the database query only once. The second bit of code is how I do it now — SamWM, Mar 13 '12 at 11:57
@SamWM there is an important distinction between `IEnumberable` and `IQueryable`. However, the call to `Get` probably shouldn't hit the database **at all** if it is returning a query; can you show how `Get` is written? — Marc Gravell, Mar 13 '12 at 11:59
When you retrieve using `Get` method here, it returns a certain object. `IEnumerable` or `IQuerible` here are just references that point to that object. The tangible object will behave on `Where` as it should do. If it is querible it will generated prologed expression tree in any case. — Oybek, Mar 13 '12 at 12:01
@Oybek no, not true; the type of the **variable** (not the object) is crucial here; if it is only known to be `IEnumerable` (even if it is actually something that implements `IQueryable`), then it will use `Enumerable.Where`, which is LINQ-to-Objects. It will not "compose" the query. Sorry, but what you say is wrong. — Marc Gravell, Mar 13 '12 at 12:08
@SamWM I've added a second approach, of expression-tree re-writing; you can use that to combine two filters **before** calling Get - any use? — Marc Gravell, Mar 13 '12 at 12:16
@SamWM yes, but would need to be `filter = filter.Combine(x => x.B > 2.5);` etc - i.e. needs to be a return value that you catch. And you might want to call it `AndAlso` or something — Marc Gravell, Mar 13 '12 at 13:21
@SamWM in fact, it is probably better to use `AndAlso` in the expression itself (will edit that in) — Marc Gravell, Mar 13 '12 at 13:22

Amy B · Answer 2 · 2012-03-13T12:39:56.647

I think you want to combine filters this way:

var myFilters = new List<Expression<Func<Customer, bool>>>();
myFilters.Add(c => c.Name.StartsWith("B"));
myFilters.Add(c => c.Orders.Count() == 3);
if (stranded)
{
  myFilters.Add(c => c.Friends.Any(f => f.Cars.Any())); //friend has car
}
Expression<Func<Customer, bool>> filter = myFilters.AndTheseFiltersTogether();
IEnumerable<Customer> thoseCustomers = Data.Get(filter);

This code will allow you to combine your filters.

    public static Expression<Func<T, bool>> OrTheseFiltersTogether<T>(params Expression<Func<T, bool>>[] filters)
    {
        return filters.OrTheseFiltersTogether();
    }

    public static Expression<Func<T, bool>> OrTheseFiltersTogether<T>(this IEnumerable<Expression<Func<T, bool>>> filters)
    {
        if (!filters.Any())
        {
            Expression<Func<T, bool>> alwaysTrue = x => true;
            return alwaysTrue;
        }

        Expression<Func<T, bool>> firstFilter = filters.First();

        var body = firstFilter.Body;
        var param = firstFilter.Parameters.ToArray();
        foreach (var nextFilter in filters.Skip(1))
        {
            var nextBody = Expression.Invoke(nextFilter, param);
            body = Expression.OrElse(body, nextBody);
        }
        Expression<Func<T, bool>> result = Expression.Lambda<Func<T, bool>>(body, param);
        return result;
    }


    public static Expression<Func<T, bool>> AndTheseFiltersTogether<T>(params Expression<Func<T, bool>>[] filters)
    {
        return filters.AndTheseFiltersTogether();
    }

    public static Expression<Func<T, bool>> AndTheseFiltersTogether<T>(this IEnumerable<Expression<Func<T, bool>>> filters)
    {
        if (!filters.Any())
        {
            Expression<Func<T, bool>> alwaysTrue = x => true;
            return alwaysTrue;
        }
        Expression<Func<T, bool>> firstFilter = filters.First();

        var body = firstFilter.Body;
        var param = firstFilter.Parameters.ToArray();
        foreach (var nextFilter in filters.Skip(1))
        {
            var nextBody = Expression.Invoke(nextFilter, param);
            body = Expression.AndAlso(body, nextBody);
        }
        Expression<Func<T, bool>> result = Expression.Lambda<Func<T, bool>>(body, param);
        return result;
    }

This approach is not *wrong*, but is poorly supported by many LINQ engines; LINQ-to-SQL is fine with Expression.Invoke, however EF does not like it. As such, it is more reliable (and no extra work) to use the "visitor" approach to combine the predicates **directly**. — Marc Gravell, Mar 13 '12 at 12:39
Looked promising though, want to be as generic as I can as there are many different classes involved. Using Entity Framework, but hoping to future proof if something else would ever be used (like NHibernate). Like the idea of building up a list of filters, then just combining before executing — SamWM, Mar 13 '12 at 13:09
@SamWM ideally you should be able to combine my simple method signatures with Marc's expression visitors to get to a use-able EF solution. — Amy B, Mar 13 '12 at 14:09

Oybek · Answer 3 · 2012-03-13T12:09:22.270

0

If you Get method retrives the data and returns in memory objects the you could do so

Expression<Func<Project, bool>> filter = (Project p) => p.UserName == "Bob";
if(showArchived) {
     filter = (Project p) => p.UserName == "Bob" && p.Archived;
}
IEnumerable<Project> projects = unitOfWork.ProjectRepository.Get(filter);

EDIT

Just to point out. When you use .ToList() method it enumerates the Queryable, i.e. makes a database request.

edited Mar 13 '12 at 12:09

answered Mar 13 '12 at 12:03

Oybek

7,016
5
29
49

That adds extra redundant code. The initial filter may be more than just checking the user name. – SamWM Mar 13 '12 at 12:07

score 0 · Answer 4 · answered Mar 13 '12 at 12:08

This all depends on how does ProjectRepository.Get() behave and what it returns. The usual way (for example, LINQ to SQL does something like this) is that it returns a IQueryable<T> and lets you (among other things) add more Where() clauses before sending it to the server in the form of one SQL query, with all the Where() clauses included. If this is the case, Mark's solution (use IQuerybale<T>) will work for you.

But if the Get() method executes the query based on the filter immediately, you need to pass it the whole filter in the expression. To do that, you can use PredicateBuilder.

score 0 · Answer 5 · answered Mar 13 '12 at 12:11

0

Get rid of ToList() and you'll be just fine.

answered Mar 13 '12 at 12:11

clearpath

916
8
24

How can you update a Linq Expression with additional parameters?

5 Answers5

Linked