C#: An item with the same key has already been added, when compiling expression

Question

Ok, here's a tricky one. Hopefully there is an expression guru here who can spot what I am doing wrong here, cause I am just not getting it.

I am building up expressions that I use to filter queries. To ease that process I have a couple of Expression<Func<T, bool>> extension methods that makes my code cleaner and so far they have been working nicely. I have written tests for all of them except one, which I wrote one for today. And that test fails completely with an ArgumentException with a long stack trace. And I just don't get it. Especially since I have been using that method for a while successfully in my queries!

Anyways, here is the stack trace I get when running the test:

failed: System.ArgumentException : An item with the same key has already been added.
    at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
    at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
    at System.Linq.Expressions.ExpressionCompiler.PrepareInitLocal(ILGenerator gen, ParameterExpression p)
    at System.Linq.Expressions.ExpressionCompiler.GenerateInvoke(ILGenerator gen, InvocationExpression invoke, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.Generate(ILGenerator gen, Expression node, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateBinary(ILGenerator gen, BinaryExpression b, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.Generate(ILGenerator gen, Expression node, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateUnliftedAndAlso(ILGenerator gen, BinaryExpression b)
    at System.Linq.Expressions.ExpressionCompiler.GenerateAndAlso(ILGenerator gen, BinaryExpression b, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateBinary(ILGenerator gen, BinaryExpression b, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.Generate(ILGenerator gen, Expression node, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateUnliftedOrElse(ILGenerator gen, BinaryExpression b)
    at System.Linq.Expressions.ExpressionCompiler.GenerateOrElse(ILGenerator gen, BinaryExpression b, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateBinary(ILGenerator gen, BinaryExpression b, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.Generate(ILGenerator gen, Expression node, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateInvoke(ILGenerator gen, InvocationExpression invoke, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.Generate(ILGenerator gen, Expression node, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateUnliftedAndAlso(ILGenerator gen, BinaryExpression b)
    at System.Linq.Expressions.ExpressionCompiler.GenerateAndAlso(ILGenerator gen, BinaryExpression b, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateBinary(ILGenerator gen, BinaryExpression b, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.Generate(ILGenerator gen, Expression node, StackType ask)
    at System.Linq.Expressions.ExpressionCompiler.GenerateLambda(LambdaExpression lambda)
    at System.Linq.Expressions.ExpressionCompiler.CompileDynamicLambda(LambdaExpression lambda)
    at System.Linq.Expressions.Expression`1.Compile()
    PredicateTests.cs(257,0): at Namespace.ExpressionExtensionsTests.WhereWithin_CollectionIsFilteredAsExpected()

The test itself looks like the following, an it fails at the Compile statement:

[Test]
public void WhereWithin_CollectionIsFilteredAsExpected()
{
    var range = new[] { Range.Create(2, 7), Range.Create(15, 18) };

    var predicate = Predicate
        .Create<int>(x => x % 2 == 0)
        .AndWithin(range, x => x)
        .Compile();

    var actual = Enumerable.Range(0, 20)
        .Where(predicate)
        .ToArray();

    Assert.That(actual, Is.EqualTo(new[] { 2, 4, 6, 16, 18 }));
}

I just don't understand the error message. I thought it might have to do with the fact that I always use x as the parameter name, but didn't seem to help when I tried to swap them around. What makes it even weirder to me is that I have been using this exact method for a while already in bigger Linq2Sql queries and they have always worked nicely. So in my test I tried to not compile the expression and use AsQueryable so I could use it on that instead. But that just made the exception occur on the ToArray instead. What is going on here? How can I fix this?

You can find the offending and annoying code in the zip file below the line:

---

Note: I had posted some of the related code here, but after some comments I decided to extract the code into it's own project which shows the exception more clearly. And more importantly, that can be run, compiled and debugged.

• ExpressionCuriosity.zip

---

Update: Simplified the example project even further with some of the suggestions from @Mark. Like removing the range class and instead just hard coding single constant range. Also added another example where using the exact same method actually works fine. So, using the AndWithin method makes the app crash, while using the WhereWithin method actually works fine. I feel pretty much clueless!

• ExpressionCuriosity.zip (Updated)

Answer 1

I refactored your methods a bit to make the compiler a bit happier:

public static Expression<Func<TSubject, bool>> AndWithin<TSubject, TField>(
    this Expression<Func<TSubject, bool>> original,
    IEnumerable<Range<TField>> range, Expression<Func<TSubject, TField>> field) where TField : IComparable<TField>
{
  return original.And(range.GetPredicateFor(field));
}


static Expression<Func<TSource, bool>> GetPredicateFor<TSource, TValue>
    (this IEnumerable<Range<TValue>> range, Expression<Func<TSource, TValue>> selector) where TValue : IComparable<TValue>
{
  var param = Expression.Parameter(typeof(TSource), "x");

  if (range == null || !range.Any())
    return Expression.Lambda<Func<TSource, bool>>(Expression.Constant(false), param);

  Expression body = null;
  foreach (var r in range)
  {
    Expression<Func<TValue, TValue, TValue, bool>> BT = (val, min, max) => val.CompareTo(min) >= 0 && val.CompareTo(max) <= 0;
    var newPart = Expression.Invoke(BT, param,
                                      Expression.Constant(r.Start, typeof(TValue)),
                                      Expression.Constant(r.End, typeof(TValue)));

    body = body == null ? newPart : (Expression)Expression.OrElse(body, newPart);
  }

  return Expression.Lambda<Func<TSource, bool>>(body, param);
}

Both have the added restriction of IComparable<TValue> (the only change to the first method).

In the second, I'm doing the comparison via a Func Expression implementation, notice that the func is created inside the loop...it's the second addition of this (what it thinks is the same...) expression in the old method that's blowing up.

Disclaimer: I still don't fully understand why your previous method didn't work, but this alternative approach bypasses the problem. Let me know if this isn't what you're after and we'll try something else.

Also, kudos on ASKING a question well, a sample project is exemplary.

Answer 2

This is not an answer, but I hope it will help someone find the answer. I've simplified the code further so that it is just one single file and still fails in the same way. I have renamed the variables so that "x" is not used twice. I have removed the Range class and replaced it with hardcoded constants 0 and 1.

using System;
using System.Linq;
using System.Linq.Expressions;

class Program
{
    static Expression<Func<int, bool>> And(Expression<Func<int, bool>> first,
                                           Expression<Func<int, bool>> second)
    {
        var x = Expression.Parameter(typeof(int), "x");
        var body = Expression.AndAlso(Expression.Invoke(first, x), Expression.Invoke(second, x));
        return Expression.Lambda<Func<int, bool>>(body, x);
    }

    static Expression<Func<int, bool>> GetPredicateFor(Expression<Func<int, int>> selector)
    {
        var param = Expression.Parameter(typeof(int), "y");
        var member = Expression.Invoke(selector, param);

        Expression body =
            Expression.AndAlso(
                Expression.GreaterThanOrEqual(member, Expression.Constant(0, typeof(int))),
                Expression.LessThanOrEqual(member, Expression.Constant(1, typeof(int))));

        return Expression.Lambda<Func<int, bool>>(body, param);
    }

    static void Main()
    {
        Expression<Func<int, bool>> predicate = a => true;
        predicate = And(predicate, GetPredicateFor(b => b)); // Comment out this line and it will run without error
        var z = predicate.Compile();
    }
}

The expression looks like this in the debugger:

x => (Invoke(a => True,x) && Invoke(y => ((Invoke(b => b,y) >= 0) && (Invoke(b => b,y) <= 1)),x))

Update: I've simplified it down to about the most simple it can be while still throwing the same exception:

using System;
using System.Linq;
using System.Linq.Expressions;

class Program
{
    static void Main()
    {
        Expression<Func<int, bool>> selector = b => true;
        ParameterExpression param = Expression.Parameter(typeof(int), "y");
        InvocationExpression member = Expression.Invoke(selector, param);
        Expression body = Expression.AndAlso(member, member);
        Expression<Func<int, bool>> predicate = Expression.Lambda<Func<int, bool>>(body, param);
        var z = predicate.Compile();
    }
}