How to implement custom LINQ Provider for decorator?

Question

I have a project that allows for some computed properties and business logic implemented via decorator and interfaces that controls all access to an EF Code first layer. I want to expose this business logic layer via oData and allow standard IQueryable functionality to filter, order and page. I need the query to be applied on the db level and not just resulting IEnumerable via Linq to Objects query for various reasons.

My class structure looks something like LogicClass(Repository) > Interface > Decorator > Poco. And the classes look something like:

public class PeopleLogicLayer
{
    // ... business and query  logic ...

    // basic query used internally
    private System.Linq.IQueryable<PersonEfPoco> GetQuery()
    {
        if (this.CurrentQuery == null) this.ResetQuery();

        var skipQuantity = (this.Page <= 1) ? 0 : (this.Page - 1) * this.PageSize;

        return this.CurrentQuery.Skip(skipQuantity)
                    .Take(this.PageSize)
                    .AsQueryable();
    }
}

public interface IPerson
{
    int Id { get; set; }
    String FirstName { get; set; }
    String LastName { get; set; }
    String FullName { get; }
}

public class PersonEfPoco
{
    public int Id { get; set; }
    public String FirstName { get; set; }

    public String LastName { get; set; }
}

public class PersonDecorator : IPerson
{
    private PersonEfPoco _person;

    public PersonDecorator(PersonEfPoco person)
    {
        this._person = person;
    }

    public int Id
    {
        get { return this._person.Id; }
        set { this._person.Id = value; }
    }

    public String FirstName
    {
        get { return this._person.FirstName; }
        set { this._person.FirstName = value; }
    }

    public String LastName
    {
        get { return this._person.LastName }
        set { this._person.LastName = value }
    }

    public String FullName
    {
        get { return $"{this._person.FirstName} {this._person.LastName}"; }
    }
}

What I want to be able to do is something like:

List<IPerson> peopleNamedBob = 
    from o in (new PeopleHiddenBehindBusinessLogic()) where o.FirstName == "Bob" select o;

or

List<IPerson> peopleNamedBob = 
    (new PeopleHiddenBehindBusinessLogic()).Where(o => o.FirstName == "Bob").ToList();

This is an over simplification. It is not really possible to do a in-query conversion via 'select new PersonDecorator(o)' since there is complex logic in the decorator layer and this is handled in there, and we don't want to allow access directly to the EF layer, instead preferring to keep the queries on the more abstract layer of the decorator.

I have considered going down the path of implementing a custom Linq provider from scratch like is mentioned here. However that article is very dated and I would think there is a better way in the last 5 years. I found re-linq which sounds like it has potential. However, when I search for tutorials for re-linq there isn't much to be had.

From what I can gather, the high level steps are to create a visitor to replace the subject of the query and convert the filter expressions to match the poco (if it can, most the property names will match) and pass it on to EF. Then save off the expressions not compatible with the EF Poco to later filter the final decorated collection. (purposely leaving out the complications of paging for now)

UPDATE I recently found the fluid method of supporting Linq but I am still lacking information about how to break down a 'Where' expression with the goal of using filters for IPerson on PersonEfPoco.

This brings me to my choices.

1. completely custom Linq provider like this
2. use re-linq - could use help finding a tutorial
3. or does more recent Linq offer a more precise method of implementation

So what is the most up-to-date method?

Answer 1

re-linq is great for implementing LINQ providers that translate queries into another representation, such as SQL or other query languages (disclaimer: I'm one of the original authors). You can also use it to implement a provider that "just" wants a model of the query that's easier to understand than the Expression AST generated by the C# compiler, but your mileage may vary if you actually need your result to look a lot like the original Expression AST.

About re-linq resources, there's the (dated, but still mostly okay) CodeProject sample, my old blog and the mailing list.

For your scenario, I'd like to suggest a fourth option which might be simpler than the first two (and no, current LINQ doesn't offer easier methods of provider implementation): Provide your own versions of the LINQ query operator methods.

I.e., create a DecoratorLayerQuery<...> class that, while not implementing IEnumerable<T> or IQueryable<T>, defines the query operators you need (Where, Select, SelectMany, etc.). These could then construct an underlying LINQ query on your real data source. Because C# will use any Where, Select, etc. method it finds, this will work as good as with "real" enumerables.

Here's what I mean:

public interface IDecoratorLayerQuery<TDecorated>
{
  IDecoratorLayerQuery<TDecorated> Where (Expression<Func<TDecorated, bool>> predicate);
  // etc.      
}

public class DecoratorLayerQuery<TDecorated, TUnderlying> : IDecoratorLayerQuery<TDecorated>
{
  private IQueryable<TUnderlying> _underlyingQuery;

  public DecoratorLayerQuery(IQueryable<TUnderlying> underlyingQuery)
  {
    _underlyingQuery = underlyingQuery;
  }

  public IDecoratorLayerQuery<TDecorated> Where (Expression<Func<TDecorated, bool>> predicate)
  {
    var newUnderlyingQuery = _underlyingQuery.Where(TranslateToUnderlying(predicate));
    return new DecoratorLayerQuery<TDecorated, TUnderlying> (newUnderlyingQuery);
  }

  private Expression<Func<TUnderlying, bool>> TranslateToUnderlying(Expression<Func<TDecorated, bool>> predicate)
  {
    var decoratedParameter = predicate.Parameters.Single();
    var underlyingParameter = Expression.Parameter(typeof(TUnderlying), decoratedParameter.Name + "_underlying");

    var bodyWithUnderlyingParameter = ReplaceDecoratedItem (decoratedParameter, underlyingParameter, predicate.Body);

    return Expression.Lambda<Func<TUnderlying, bool>> (bodyWithUnderlyingParameter, underlyingParameter);
  }

  private Expression ReplaceDecoratedItem(Expression decorated, Expression underlying, Expression body)
  {
    // Magic happens here: Implement an expression visitor that iterates over body and replaces all occurrences with _corresponding_ occurrences of _underlying_.
    // This will probably involve translating member expressions as well. E.g., if decorated is of type IPerson, decorated.FullName must instead become 
    // the Expression equivalent of 'underlying.FirstName + " " + underlying.FullName'.
  }

  public List<TDecorated> ToList() // And AsEnumerable, AsQueryable, etc.
  {
    var projection = /* construct Expression that transforms TUnderlying to TDecorated here */;
    return _underlyingQuery.Select(projection).ToList();
  }
}

public static class DecoratorLayerQueryFactory
{
  public static IDecoratorLayerQuery<TDecorated> CreateQuery<TDecorated>()
  {
    var underlyingType = /* calculate underlying type for TDecorated here */;
    var queryType = typeof (DecoratorLayerQuery<,>).MakeGenericType (typeof (TDecorated), underlyingType);

    var initialSource = DbContext.Set(underlyingType);
    return (IDecoratorLayerQuery<TDecorated>) Activator.CreateInstance (queryType, initialSource);
  }
}

var exampleQuery =
    from p in DecoratorLayerQueryFactory.CreateQuery<IPerson>
    where p.FullName == "John Doe"
    select p.FirstName;

The TranslateToUnderlying and ReplaceDecoratedItem methods are the real difficulty in this approach because they need to know how to (and generate expressions that) translate what the programmer wrote to what EF understands. As an extended version, they might also extract some query stuff to be executed in-memory. However, this is the essential complexity of your endeavor :)

Some additional (IMO accidental) complexity will raise its ugly head when you need to support subqueries, e.g., a Where predicate containing another query. In these cases, I'd suggest taking a look at how re-linq detects and handles such situations. If you can avoid this feature, I'd suggest you do so.