Extending Query Expressions

Question

Are there any documents or examples out there on how one can extend/add new keywords to query expressions? Is this even possible?

For example, I'd like to add a lead/lag operator.

Take a look at [this question](http://stackoverflow.com/questions/9565986/defining-new-keywords-in-fs-computation-expression). The last link in my answer may be of your interest. — pad, Dec 11 '12 at 21:24

Tomas Petricek · Accepted Answer · 2012-12-11T22:18:05.197

In addition to the query builder for the Rx Framework mentioned by @pad, there is also a talk by Wonseok Chae from the F# team about Computation Expressions that includes query expressions. I'm not sure if the meeting was recorded, but there are very detailed slides with a cool example on query syntax for generating .NET IL code.

The source code of the standard F# query builder is probably the best resource for finding out what types of operations are supported and how to annotate them with attributes.

The key attributes that you'll probably need are demonstrated by the where clause:

[<CustomOperation("where",MaintainsVariableSpace=true,AllowIntoPattern=true)>]
member Where : 
  : source:QuerySource<'T,'Q> * 
    [<ProjectionParameter>] predicate:('T -> bool) -> QuerySource<'T,'Q>

The CustomOperation attribute defines the name of the operation. The (quite important) parameter MaintainsVariableSpace allows you to say that the operation returns the same type of values as it takes as the input. In that case, the variables defined earlier are still available after the operation. For example:

query { for p in db.Products do
        let name = p.ProductName
        where (p.UnitPrice.Value > 100.0M)
        select name }

Here, the variables p and name are still accessible after where because where only filters the input, but it does not transform the values in the list.

Finally, the ProjectionParameter allows you to say that p.UnitValue > 100.0M should actually be turned into a function that takes the context (available variables) and evaluates this expression. If you do not specify this attribute, then the operation just gets the value of the argument as in:

query { for p in .. do
        take 10 }

Here, the argument 10 is just a simple expression that cannot use values in p.

It's perhaps worth mentioning that these can be extension methods on the standard query builder type. — kvb, Dec 11 '12 at 22:39
Very cool! Thanks for the links. Interesting that one can use existing linq extensions to accomplish what they want here as well. — Dave, Dec 11 '12 at 22:55
@kvb That is very cool! But then you also need to add that they won't work with the standard providers for LINQ to SQL and LINQ to Entities :-( (unless you write some pre-processor that transforms them to something else that is supported in the expression tree?) — Tomas Petricek, Dec 11 '12 at 23:14
Right, they'll only work with IEnumerables, not IQueryables. This can be enforced by making sure that the input and output types are more specific (see my comment on davewolfs's other answer). — kvb, Dec 12 '12 at 16:01

Dave · Answer 2 · 2012-12-12T17:06:16.727

6

Pretty cool feature for the language. Just implemented the reverse to query QuerySource.

Simple example, but just a demonstration.

module QueryExtensions

type ExtendedQueryBuilder() =
    inherit Linq.QueryBuilder()
    /// Defines an operation 'reverse' that reverses the sequence    
    [<CustomOperation("reverse", MaintainsVariableSpace = true)>]
    member __.Reverse (source : Linq.QuerySource<'T,System.Collections.IEnumerable>) =
        let reversed = source.Source |> List.ofSeq |> List.rev
        new Linq.QuerySource<'T,System.Collections.IEnumerable>(reversed)


let query = ExtendedQueryBuilder()

And now it being used.

let a = [1 .. 100]

let specialReverse = 
    query {
        for i in a do
        select i
        reverse
    }

edited Dec 12 '12 at 17:06

answered Dec 11 '12 at 22:44

Dave

2,386
1
20
38

Note that you may want to use `System.Collections.IEnumerable` in both places where you've got `'Q`, since the standard IQueryable providers don't know how to handle your operation and will throw a runtime exception if it's used. – kvb Dec 12 '12 at 16:00
It seems to run fine, but will add. Can you explain that in more detail? What do they know how to handle? – Dave Dec 12 '12 at 17:05
Basically, IQueryable providers work against a quoted version of the query, and only recognize the built-in operators. So if you did something like `query { for x in myDatabaseTable do reverse }` where `myDatabaseTable` is a LINQ table, then the query would fail at runtime. – kvb Dec 12 '12 at 17:47
1

If you restrict the `'Q` type parameter to `IEnumerable`, then the query will succeed at runtime, but run all of the logic on the client side (rather than trying to generate a server-side query). – kvb Dec 12 '12 at 17:48

Extending Query Expressions

2 Answers2