Handling null values building expressions with child items

Question

We are using Expressions to build a refined select statement (being used by GraphQL .Net) that only queries the properties asked for.

We parse the raw query and use a recursive method to build an expression that we use to generate our select (via entity framework)

Our code is HEAVILY based on this answer provided by @Svyatoslav Danyliv.

Our expressionbuilder class looks like this:

public class ExpressionBuilder
{
    public Expression<Func<TSource, TTarget>> BuildSelector<TSource, TTarget>(string members)
    {
        return BuildSelector<TSource, TTarget>(members.Split(',').Select(m => m.Trim()));
    }

    public Expression<Func<TSource, TTarget>> BuildSelector<TSource, TTarget>(IEnumerable<string> members)
    {
        var parameter = Expression.Parameter(typeof(TSource), "e");
        var body = NewObject(typeof(TTarget), parameter, members.Select(m => m.Split('.')));
        return Expression.Lambda<Func<TSource, TTarget>>(body, parameter);
    }

    private Expression NewObject(Type targetType, Expression source, IEnumerable<string[]> memberPaths, int depth = 0)
    {
        var bindings = new List<MemberBinding>();
        var target = Expression.Constant(null, targetType);
        foreach (var memberGroup in memberPaths.GroupBy(path => path[depth]))
        {
            var memberName = memberGroup.Key;
            var targetMember = Expression.PropertyOrField(target, memberName);
            var sourceMember = Expression.PropertyOrField(source, memberName);
            var childMembers = memberGroup.Where(path => depth + 1 < path.Length).ToList();

            Expression targetValue = null;
            if (!childMembers.Any())
            {
                targetValue = sourceMember;
            }
            else
            {
                if (IsEnumerableType(targetMember.Type, out var sourceElementType) &&
                    IsEnumerableType(targetMember.Type, out var targetElementType))
                {
                    var sourceElementParam = Expression.Parameter(sourceElementType, "e");
                    targetValue = NewObject(targetElementType, sourceElementParam, childMembers, depth + 1);
                    targetValue = Expression.Call(typeof(Enumerable), nameof(Enumerable.Select),
                        new[] { sourceElementType, targetElementType }, sourceMember,
                        Expression.Lambda(targetValue, sourceElementParam));

                    targetValue = CorrectEnumerableResult(targetValue, targetElementType, targetMember.Type);
                }
                else
                {
                    targetValue = NewObject(targetMember.Type, sourceMember, childMembers, depth + 1);
                }
            }

            bindings.Add(Expression.Bind(targetMember.Member, targetValue));
        }
        return Expression.MemberInit(Expression.New(targetType), bindings);
    }

    private bool IsEnumerableType(Type type, out Type elementType)
    {
        foreach (var intf in type.GetInterfaces())
        {
            if (intf.IsGenericType && intf.GetGenericTypeDefinition() == typeof(IEnumerable<>))
            {
                elementType = intf.GetGenericArguments()[0];
                return true;
            }
        }

        elementType = null;
        return false;
    }

    private bool IsSameCollectionType(Type type, Type genericType, Type elementType)
    {
        var result = genericType.MakeGenericType(elementType).IsAssignableFrom(type);
        return result;
    }

    private Expression CorrectEnumerableResult(Expression enumerable, Type elementType, Type memberType)
    {
        if (memberType == enumerable.Type)
            return enumerable;

        if (memberType.IsArray)
            return Expression.Call(typeof(Enumerable), nameof(Enumerable.ToArray), new[] { elementType }, enumerable);

        if (IsSameCollectionType(memberType, typeof(List<>), elementType)
            || IsSameCollectionType(memberType, typeof(ICollection<>), elementType)
            || IsSameCollectionType(memberType, typeof(IReadOnlyList<>), elementType)
            || IsSameCollectionType(memberType, typeof(IReadOnlyCollection<>), elementType))
            return Expression.Call(typeof(Enumerable), nameof(Enumerable.ToList), new[] { elementType }, enumerable);

        throw new NotImplementedException($"Not implemented transformation for type '{memberType.Name}'");
    }
}

and the usage is as follows:

var builder = new ExpressionBuilder();
var statement = builder.BuildSelector<Payslip, Payslip>(selectStatement);

The select statement above looks like "id,enrollment.id" where we are selecting parentobject.id and parentobject.enrollment.id

This statement variable is then used in a EntityQueryable.Select(statement) against a filtered database collection there we have essentially done the 'where' aspect of the query.

When working correctly the sql generated is ONLY the fields requested, and it works great. However in this instance it is trying to access the .Id property of a null value and constantly fails.

It is currently failing because the "Enrollment" property of the parentObject is not set, thus null.

We get the following error

System.InvalidOperationException: Nullable object must have a value.
   at System.Nullable`1.get_Value()
   at lambda_method2198(Closure , QueryContext , DbDataReader , ResultContext , SingleQueryResultCoordinator )
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()

I've tried to implement a try catch expression but keep failing, and end up with the exact same error. At this time, we aren't hitting the database, so we don't know if the value will be null or not (it works fine if it's not null).

I am by no means an expert on expressions, and am really struggling to find just where I should be wrapping this value, or even how. Ideally I'd just like to return null, but worst case a default enrollment object could be detected client side and possibly handled ok.

** UPDATE!! ** Reading into this I think that this is a result of this issue. The workaround would involve building something like this null check into the expression builder.

.Select(r => new Something()
    {
        X1 = r.X1,
        X2 = r.X2,

        X = new X() 
        {
            Name= r.X.Name
        },
        XX = r.XXNullableId == null ? null : new XX()
        {
            XNumber = r.XX.XNumber
        }
    })

I have created a VERY simple EF Core Application here that creates a simple three table database and recreates the problem.

I believe the workaround mentioned above will work, it's just a matter of figuring out how to add it to the expression builder.

Answer 1

So I figured this one out (sort of) by special casing the property that causes the issue.

Due to the nature of our model, this situation will only occur in a few places, it's easier to add some workarounds for them, that have the expression builder do some strange null check on every property.

I created a simple attribute that I put on the 'Object' Property and have it point to the property I need to check for null.

That looks like this:

[System.AttributeUsage(System.AttributeTargets.Property, Inherited = true, AllowMultiple = true)]
public class ExpressionBuilderNullCheckForPropertyAttribute : Attribute
{
    public ExpressionBuilderNullCheckForPropertyAttribute(string propertyName)
    {
        this.PropertyName = propertyName;
    }
    public string PropertyName { get; set; }
}

So now, my 'Person' Class (from the linked example repo above) now looks like this.

public class Person
{
    [Required]
    [Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public Guid Id { get; set; }
    public string Name { get; set; }
    public int Age { get; set; }
    
    public Nullable<Guid> VetId { get; set; }
    
    [ExpressionBuilderNullCheckForProperty(nameof(VetId))]
    public Vet Vet { get; set; }
    
    public List<Pet> Pets { get; set; }
}

Then I added a check into the ExpressionBuilder class to check for that attribute, if it is present, lookup the property it has mentioned, then perform a null check. If that property is null, then return a default object for that type. Which, in this case, is null.

if (targetMember.Member.GetCustomAttributesData().Any(x => x.AttributeType == typeof(ExpressionBuilderNullCheckForPropertyAttribute)))
{
    var att = (ExpressionBuilderNullCheckForPropertyAttribute)targetMember.Member.GetCustomAttributes(true).First(x => x.GetType() == typeof(ExpressionBuilderNullCheckForPropertyAttribute));
    var propertyToCheck = att.PropertyName;
    
    var valueToCheck = Expression.PropertyOrField(source, propertyToCheck);
    var nullCheck = Expression.Equal(valueToCheck, Expression.Constant(null, targetMember.Type));
    
    var checkForNull = Expression.Condition(nullCheck, Expression.Default(targetMember.Type), targetValue);
    bindings.Add(Expression.Bind(targetMember.Member, checkForNull));
}
else
{
    bindings.Add(Expression.Bind(targetMember.Member, targetValue));
}

I have updated the example repo if anyone wants to see the fix working.