2

note: it's a long post, please scroll to the bottom to see questions - hopefully that will make it easier to understand my problem. Thanks!


I have "Member" model which is defined as follows:

public class Member
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string ScreenName { get; set; }

    [NotMapped]
    public string RealName
    {
         get { return (FirstName + " " + LastName).TrimEnd(); }
    }

    [NotMapped]
    public string DisplayName
    {
        get
        {
            return string.IsNullOrEmpty(ScreenName) ? RealName : ScreenName;
        }
    }
}

This is existing project, and model, and I don't want to change this. Now we got a request to enable profile retrieval by DisplayName:

public Member GetMemberByDisplayName(string displayName)
{
     var member = this.memberRepository
                      .FirstOrDefault(m => m.DisplayName == displayName);
     return member;
}

This code does not work because DisplayName is not mapped to a field in database. Okay, I will make an expression then:

public Member GetMemberByDisplayName(string displayName)
{
     Expression<Func<Member, bool>> displayNameSearchExpr = m => (
                string.IsNullOrEmpty(m.ScreenName) 
                    ? (m.Name + " " + m.LastName).TrimEnd() 
                    : m.ScreenName
            ) == displayName;

     var member = this.memberRepository
                      .FirstOrDefault(displayNameSearchExpr);

     return member;
}

this works. The only problem is that business logic to generate display name is copy/pasted in 2 different places. I want to avoid this. But I do not understand how to do this. The best I came with is the following:

  public class Member
    {

        public static Expression<Func<Member, string>> GetDisplayNameExpression()
        {
            return m => (
                            string.IsNullOrEmpty(m.ScreenName)
                                ? (m.Name + " " + m.LastName).TrimEnd()
                                : m.ScreenName
                        );
        }

        public static Expression<Func<Member, bool>> FilterMemberByDisplayNameExpression(string displayName)
        {
            return m => (
                string.IsNullOrEmpty(m.ScreenName)
                    ? (m.Name + " " + m.LastName).TrimEnd()
                    : m.ScreenName
            ) == displayName;
        }

        private static readonly Func<Member, string> GetDisplayNameExpressionCompiled = GetDisplayNameExpression().Compile();

        [NotMapped]
        public string DisplayName
        {
            get
            {
                return GetDisplayNameExpressionCompiled(this);
            }
        }

        [NotMapped]
        public string RealName
        {
             get { return (FirstName + " " + LastName).TrimEnd(); }
        }

   }

Questions:

(1) How to reuse GetDisplayNameExpression() inside FilterMemberByDisplayNameExpression()? I tried Expression.Invoke:

public static Expression<Func<Member, bool>> FilterMemberByDisplayNameExpression(string displayName)
{
    Expression<Func<string, bool>> e0 = s => s == displayName;
    var e1 = GetDisplayNameExpression();

    var combinedExpression = Expression.Lambda<Func<Member, bool>>(
           Expression.Invoke(e0, e1.Body), e1.Parameters);

    return combinedExpression;
}

but I get the following error from the provider:

The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.

(2) Is it a good approach to use Expression.Compile() inside DisplayName property? Any issues with it?

(3) How to move RealName logic inside GetDisplayNameExpression()? I think I have to create another expression and another compiled expression, but I do not understand how to CALL RealNameExpression from inside GetDisplayNameExpression().

Thank you.

avs099
  • 10,937
  • 6
  • 60
  • 110
  • A) There are no good solutions. B) What you have done is a nightmare (not because you don't know what you are doing, but because the `FilterMemberByDisplayNameExpression` is a nightmare waiting to be dreamed). C) There are no good solutions. – xanatos Aug 28 '13 at 12:48
  • 1
    @xanatos why is (B) a nightmare? Can you please explain what could be the biggest problem? – avs099 Aug 28 '13 at 12:51
  • If you really really want to do something, you could create a "filter" like this: `static IQueryable MakeMyExpressionWork(this IQueryable exp)`. This magic filter, through expression rewriting, reflection, mumbo jumbo etc would modify the `Expression`. – xanatos Aug 28 '13 at 12:53
  • @xanatos what you said is true for (A) and (C) cases - I understand that I need to modify expression somehow, that was my question. But I'm confused about (B) - what's wrong with compiling expression and using it inside property, just to avoid business logic copy/paste? – avs099 Aug 28 '13 at 12:55
  • To save you 5 lines of code duplication, you have to write 20 lines of Expression/Expression compilation etc... And the support for Expression "connection" isn't very strong. Let's say that your `FilterMemberByDisplayNameExpression` works... How do you put it in an `or` (`||`) expression? You'll have to create an `or` operator that accepts `Expressions`... Then tomorrow they'll ask you "can we do a Contain on the `DisplayName`? – xanatos Aug 28 '13 at 12:55
  • No, my A, B, C weren't connected to your questions (that are numbered 1, 2, 3)... My response is "no good solutions possible to what you want to do, you are creating a nightmare, no good solutions possible". – xanatos Aug 28 '13 at 12:56
  • well, you misunderstood the point then. I MUST do the expression because this should be executed on database level, right? So there is no question that it should be done. What i was trying to avoid is business logic copy/paste. And that a bit simplified example example of course. – avs099 Aug 28 '13 at 13:05
  • 1
    If you have Sql Server as db, would you mind using Computed columns ? This would make these kind of things really easier (we faced the same kind of problem little time ago) ! – Raphaël Althaus Aug 28 '13 at 13:13
  • @avs099 And no, I didn't misunderstood your problem. But I know that *The road to hell is paved with good intentions*. – xanatos Aug 28 '13 at 13:22
  • @RaphaëlAlthaus - that won't work - as I said, I do not have the flexibility to change existing project/code a lot. And what you are suggesting would be business logic duplication in this case. But thanks for the tip! – avs099 Aug 28 '13 at 13:23
  • That wouldn't be duplication logic, as the code would be in one place only (computed column declaration)... But if you can't... you can't ! – Raphaël Althaus Aug 28 '13 at 13:24
  • @avs099 Added a generic solution to "handle" special properties in `IQueryable` expressions. – xanatos Aug 29 '13 at 08:30

2 Answers2

2

I can fix your expression generator, and I can compose your GetDisplayNameExpression (so 1 and 3)

public class Member
{
    public string ScreenName { get; set; }
    public string Name { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }

    public static Expression<Func<Member, string>> GetRealNameExpression()
    {
        return m => (m.Name + " " + m.LastName).TrimEnd();
    }

    public static Expression<Func<Member, string>> GetDisplayNameExpression()
    {
        var isNullOrEmpty = typeof(string).GetMethod("IsNullOrEmpty", BindingFlags.Static | BindingFlags.Public, null, new[] { typeof(string) }, null);

        var e0 = GetRealNameExpression();
        var par1 = e0.Parameters[0];

        // Done in this way, refactoring will correctly rename m.ScreenName
        // We could have used a similar trick for string.IsNullOrEmpty,
        // but it would have been useless, because its name and signature won't
        // ever change.
        Expression<Func<Member, string>> e1 = m => m.ScreenName;

        var screenName = (MemberExpression)e1.Body;
        var prop = Expression.Property(par1, (PropertyInfo)screenName.Member);
        var condition = Expression.Condition(Expression.Call(null, isNullOrEmpty, prop), e0.Body, prop);

        var combinedExpression = Expression.Lambda<Func<Member, string>>(condition, par1);
        return combinedExpression;
    }

    private static readonly Func<Member, string> GetDisplayNameExpressionCompiled = GetDisplayNameExpression().Compile();

    private static readonly Func<Member, string> GetRealNameExpressionCompiled = GetRealNameExpression().Compile();

    public string DisplayName
    {
        get
        {
            return GetDisplayNameExpressionCompiled(this);
        }
    }

    public string RealName
    {
        get
        {
            return GetRealNameExpressionCompiled(this);
        }
    }

    public static Expression<Func<Member, bool>> FilterMemberByDisplayNameExpression(string displayName)
    {
        var e0 = GetDisplayNameExpression();
        var par1 = e0.Parameters[0];

        var combinedExpression = Expression.Lambda<Func<Member, bool>>(
            Expression.Equal(e0.Body, Expression.Constant(displayName)), par1);

        return combinedExpression;
    }

Note how I reuse the same parameter of the GetDisplayNameExpression expression e1.Parameters[0] (put in par1) so that I don't have to rewrite the expression (otherwise I would have needed to use an expression rewriter).

We could use this trick because we had only a single expression to handle, to which we had to attach some new code. Totally different (we would have needed an expression rewriter) would be the case of trying to combine two expressions (for example to do a GetRealNameExpression() + " " + GetDisplayNameExpression(), both require as a parameter a Member, but their parameters are separate... Probably this https://stackoverflow.com/a/5431309/613130 would work...

For the 2, I don't see any problem. You are correctly using static readonly. But please, look at GetDisplayNameExpression and think "is it better some business code duplication pay or that?"

Generic solution

Now... I was quite sure it was doable... and in fact it is doable: an expression "expander" that "expands" "special properties" to their Expression(s) "automagically".

public static class QueryableEx
{
    private static readonly ConcurrentDictionary<Type, Dictionary<PropertyInfo, LambdaExpression>> expressions = new ConcurrentDictionary<Type, Dictionary<PropertyInfo, LambdaExpression>>();

    public static IQueryable<T> Expand<T>(this IQueryable<T> query)
    {
        var visitor = new QueryableVisitor();
        Expression expression2 = visitor.Visit(query.Expression);

        return query.Expression != expression2 ? query.Provider.CreateQuery<T>(expression2) : query;
    }

    private static Dictionary<PropertyInfo, LambdaExpression> Get(Type type)
    {
        Dictionary<PropertyInfo, LambdaExpression> dict;

        if (expressions.TryGetValue(type, out dict))
        {
            return dict;
        }

        var props = type.GetProperties(BindingFlags.Public | BindingFlags.Instance);

        dict = new Dictionary<PropertyInfo, LambdaExpression>();

        foreach (var prop in props)
        {
            var exp = type.GetMember(prop.Name + "Expression", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static).Where(p => p.MemberType == MemberTypes.Field || p.MemberType == MemberTypes.Property).SingleOrDefault();

            if (exp == null)
            {
                continue;
            }

            if (!typeof(LambdaExpression).IsAssignableFrom(exp.MemberType == MemberTypes.Field ? ((FieldInfo)exp).FieldType : ((PropertyInfo)exp).PropertyType))
            {
                continue;
            }

            var lambda = (LambdaExpression)(exp.MemberType == MemberTypes.Field ? ((FieldInfo)exp).GetValue(null) : ((PropertyInfo)exp).GetValue(null, null));

            if (prop.PropertyType != lambda.ReturnType)
            {
                throw new Exception(string.Format("Mismatched return type of Expression of {0}.{1}, {0}.{2}", type.Name, prop.Name, exp.Name));
            }

            dict[prop] = lambda;
        }

        // We try to save some memory, removing empty dictionaries
        if (dict.Count == 0)
        {
            dict = null;
        }

        // There is no problem if multiple threads generate their "versions"
        // of the dict at the same time. They are all equivalent, so the worst
        // case is that some CPU cycles are wasted.
        dict = expressions.GetOrAdd(type, dict);

        return dict;
    }

    private class SingleParameterReplacer : ExpressionVisitor
    {
        public readonly ParameterExpression From;
        public readonly Expression To;

        public SingleParameterReplacer(ParameterExpression from, Expression to)
        {
            this.From = from;
            this.To = to;
        }

        protected override Expression VisitParameter(ParameterExpression node)
        {
            return node != this.From ? base.VisitParameter(node) : this.Visit(this.To);
        }
    }

    private class QueryableVisitor : ExpressionVisitor
    {
        protected static readonly Assembly MsCorLib = typeof(int).Assembly;
        protected static readonly Assembly Core = typeof(IQueryable).Assembly;

        // Used to check for recursion
        protected readonly List<MemberInfo> MembersBeingVisited = new List<MemberInfo>();

        protected override Expression VisitMember(MemberExpression node)
        {
            var declaringType = node.Member.DeclaringType;
            var assembly = declaringType.Assembly;

            if (assembly != MsCorLib && assembly != Core && node.Member.MemberType == MemberTypes.Property)
            {
                var dict = QueryableEx.Get(declaringType);

                LambdaExpression lambda;

                if (dict != null && dict.TryGetValue((PropertyInfo)node.Member, out lambda))
                {
                    // Anti recursion check
                    if (this.MembersBeingVisited.Contains(node.Member))
                    {
                        throw new Exception(string.Format("Recursively visited member. Chain: {0}", string.Join("->", this.MembersBeingVisited.Concat(new[] { node.Member }).Select(p => p.DeclaringType.Name + "." + p.Name))));
                    }

                    this.MembersBeingVisited.Add(node.Member);

                    // Replace the parameters of the expression with "our" reference
                    var body = new SingleParameterReplacer(lambda.Parameters[0], node.Expression).Visit(lambda.Body);

                    Expression exp = this.Visit(body);

                    this.MembersBeingVisited.RemoveAt(this.MembersBeingVisited.Count - 1);

                    return exp;
                }
            }

            return base.VisitMember(node);
        }
    }
}
  • How does it work? Magic, reflection, fairy dust...
  • Does it support properties referencing other properties? Yes
  • What does it need?

It needs that every "special" property of name Foo has a corresponding static field/static property named FooExpression that returns an Expression<Func<Class, something>>

It needs that the query is "transformed" through the extension method Expand() at some point before the materialization/enumeration. So:

public class Member
{
    // can be private/protected/internal
    public static readonly Expression<Func<Member, string>> RealNameExpression =
        m => (m.Name + " " + m.LastName).TrimEnd();

    // Here we are referencing another "special" property, and it just works!
    public static readonly Expression<Func<Member, string>> DisplayNameExpression =
        m => string.IsNullOrEmpty(m.ScreenName) ? m.RealName : m.ScreenName;

    public string RealName
    {
        get 
        { 
            // return the real name however you want, probably reusing
            // the expression through a compiled readonly 
            // RealNameExpressionCompiled as you had done
        }  
    }

    public string DisplayName
    {
        get
        {
        }
    }
}

// Note the use of .Expand();
var res = (from p in ctx.Member 
          where p.RealName == "Something" || p.RealName.Contains("Anything") ||
                p.DisplayName == "Foo"
          select new { p.RealName, p.DisplayName, p.Name }).Expand();

// now you can use res normally.
  • Limits 1: one problem is with methods like Single(Expression), First(Expression), Any(Expression) and similar, that don't return an IQueryable. Change by using first a Where(Expression).Expand().Single()

  • Limit 2: "special" properties can't reference themselves in cycles. So if A uses B, B can't use A, and tricks like using ternary expressions won't make it work.

Community
  • 1
  • 1
xanatos
  • 109,618
  • 12
  • 197
  • 280
2

Recently I was faced with the need to keep some business logic in expressions which allow to use it in SQL queries and in .net code. I've moved some code which help with this to github repo. I've implemented the easy way to combine and reuse expressions. See my example:

public class Person
{
    public int Id { get; set; }

    public string FirstName { get; set; }
    public string LastName { get; set; }

    public int Age { get; set; }

    public Company Company { get; set; }

    public static Expression<Func<Person, string>> FirstNameExpression
    {
        get { return x => x.FirstName; }
    }

    public static Expression<Func<Person, string>> LastNameExpression
    {
        get { return x => x.LastName; }
    }

    public static Expression<Func<Person, string>> FullNameExpression
    {
        //get { return FirstNameExpression.Plus(" ").Plus(LastNameExpression); }
        // or
        get { return x => FirstNameExpression.Wrap(x) + " " + LastNameExpression.Wrap(x); }
    }

    public static Expression<Func<Person, string>> SearchFieldExpression
    {
        get
        {
            return
                p => string.IsNullOrEmpty(FirstNameExpression.Wrap(p)) ? LastNameExpression.Wrap(p) : FullNameExpression.Wrap(p);
        }
    }

    public static Expression<Func<Person, bool>> GetFilterExpression(string q)
    {
        return p => SearchFieldExpression.Wrap(p) == q;
    }
}

Extension method .Wrap() is the marker only:

public static TDest Wrap<TSource, TDest>(this Expression<Func<TSource, TDest>> expr, TSource val)
{
    throw new NotImplementedException("Used only as expression transform marker");
}

What is FullName? It's FirstName + " " + LastName where FirstName and LastName - strings. But we have expressions, it's not a real value and we need to combine these expressions. Method .Wrap(val) help us to move to a simple code. We don't need to write any composers or other visitors for expressions. All this magic is already done by method .Wrap(val) where val - parameter which will be passed to called lambda expression.

So we describe expressions using other expressions. To get the full expression need to expand all usages of Wrap method, so you need to call method Unwrap on Expression (or IQueryable). See sample:

using (var context = new Entities())
{
    var originalExpr = Person.GetFilterExpression("ivan");
    Console.WriteLine("Original: " + originalExpr);
    Console.WriteLine();

    var expr = Person.GetFilterExpression("ivan").Unwrap();
    Console.WriteLine("Unwrapped: " + expr);
    Console.WriteLine();

    var persons = context.Persons.Where(Person.GetFilterExpression("ivan").Unwrap());
    Console.WriteLine("SQL Query 1: " + persons);
    Console.WriteLine();

    var companies = context.Companies.Where(x => x.Persons.Any(Person.GetFilterExpression("abc").Wrap())).Unwrap(); // here we use .Wrap method without parameters, because .Persons is the ICollection (not IQueryable) and we can't pass Expression<Func<T, bool>> as Func<T, bool>, so we need it for successful compilation. Unwrap method expand Wrap method usage and convert Expression to lambda function.
    Console.WriteLine("SQL Query 2: " + companies);
    Console.WriteLine();

    var traceSql = persons.ToString();
}

Console output:

Original: p => (Person.SearchFieldExpression.Wrap(p) == value(QueryMapper.Exampl es.Person+<>c__DisplayClass0).q)

Unwrapped: p => (IIF(IsNullOrEmpty(p.FirstName), p.LastName, ((p.FirstName + " " ) + p.LastName)) == value(QueryMapper.Examples.Person+<>c__DisplayClass0).q)

SQL Query 1: SELECT [Extent1].[Id] AS [Id], [Extent1].[FirstName] AS [FirstName], [Extent1].[LastName] AS [LastName], [Extent1].[Age] AS [Age], [Extent1].[Company_Id] AS [Company_Id] FROM [dbo].[People] AS [Extent1] WHERE (CASE WHEN (([Extent1].[FirstName] IS NULL) OR (( CAST(LEN([Extent1].[Firs tName]) AS int)) = 0)) THEN [Extent1].[LastName] ELSE [Extent1].[FirstName] + N' ' + [Extent1].[LastName] END) = @p_linq_0

SQL Query 2: SELECT [Extent1].[Id] AS [Id], [Extent1].[Name] AS [Name] FROM [dbo].[Companies] AS [Extent1] WHERE EXISTS (SELECT 1 AS [C1] FROM [dbo].[People] AS [Extent2] WHERE ([Extent1].[Id] = [Extent2].[Company_Id]) AND ((CASE WHEN (([Exten t2].[FirstName] IS NULL) OR (( CAST(LEN([Extent2].[FirstName]) AS int)) = 0)) TH EN [Extent2].[LastName] ELSE [Extent2].[FirstName] + N' ' + [Extent2].[LastName] END) = @p_linq_0) )

So the main idea to use .Wrap() method to convert from Expression world to non-expression which provide easy way to reuse expressions.

Let me know if you need more explanations.

Ivan Bianko
  • 1,749
  • 15
  • 22