1

I have a reference database that holds coordinates of objects on the celestial grid. I want to query the database and find objects that are "close to" (within a certain angular distance of) a given point.

I've tried this query:

const double WithinOneMinute = 1.0 / 60.0;    // 1 minute of arc
var db = CompositionRoot.GetTargetDatabase();
var targets = from item in db.Targets
              where item.RightAscension.IsCloseTo(ra.Value, WithinOneMinute) 
                    && item.Declination.IsCloseTo(dec.Value, WithinOneMinute)
              select item;
var found = targets.ToList();

This fails because the LINQ query provider doesn't understand my IsCloseTo extension method, which is implemented as:

public static bool IsCloseTo(this double comparand, double comparison, double tolerance = EightDecimalPlaces)
{
    var difference = Math.Abs(comparand - comparison);
    return (difference <= tolerance); 
}

So I'm currently stuck for ideas. Anyone done anything like this?

Peter Szekeli
  • 2,712
  • 3
  • 30
  • 44
Tim Long
  • 13,508
  • 19
  • 79
  • 147

4 Answers4

2

As you already noticed, custom functions cannot be used as part of the query expression tree. So you either have to embed the function logic manually inside the query, thus introducing a lot of code duplication, or switch to method syntax and use helper methods that return the whole expression.

The later can be done manually using the methods of System.Linq.Expressions, but that's not natural and requires a lot of knowledge. Let me present you a \n easier way.

The goal would be to implement extension method like this

public static IQueryable<T> WhereIsCloseTo<T>(this IQueryable<T> source, Expression<Func<T, double>> comparand, double comparison, double tolerance = EightDecimalPlaces)
{
    return source.Where(...);
}

and use it as follows

var targets = db.Targets
    .WhereIsCloseTo(item => item.RightAscension, ra.Value, WithinOneMinute)
    .WhereIsCloseTo(item => item.Declination, dec.Value, WithinOneMinute);

Note that with this approach you cannot use &&, but chained Where produce the equivalent result.

First, let provide the expression equivalent of your original function

public static Expression<Func<double, bool>> IsCloseTo(double comparison, double tolerance = EightDecimalPlaces)
{
    return comparand => Math.Abs(comparand - comparison) >= tolerance;
}

The problem is that it can't be used directly in our method because it needs Expression<Func<T, bool>>.

Luckily that can easily be done by using a little helper utility form my answer to Define part of an Expression as a variable in c#:

public static class ExpressionUtils
{
    public static Expression<Func<TOuter, TResult>> Bind<TOuter, TInner, TResult>(this Expression<Func<TOuter, TInner>> source, Expression<Func<TInner, TResult>> resultSelector)
    {
        var body = new ParameterExpressionReplacer { source = resultSelector.Parameters[0], target = source.Body }.Visit(resultSelector.Body);
        var lambda = Expression.Lambda<Func<TOuter, TResult>>(body, source.Parameters);
        return lambda;
    }

    public static Expression<Func<TOuter, TResult>> ApplyTo<TInner, TResult, TOuter>(this Expression<Func<TInner, TResult>> source, Expression<Func<TOuter, TInner>> innerSelector)
    {
        return innerSelector.Bind(source);
    }

    class ParameterExpressionReplacer : ExpressionVisitor
    {
        public ParameterExpression source;
        public Expression target;
        protected override Expression VisitParameter(ParameterExpression node)
        {
            return node == source ? target : base.VisitParameter(node);
        }
    }
}

Now we have everything needed, so our method is implemented simple like this:

public static IQueryable<T> WhereIsCloseTo<T>(this IQueryable<T> source, Expression<Func<T, double>> comparand, double comparison, double tolerance = EightDecimalPlaces)
{
    return source.Where(IsCloseTo(comparison, tolerance).ApplyTo(comparand));
}
Community
  • 1
  • 1
Ivan Stoev
  • 195,425
  • 15
  • 312
  • 343
  • Thanks Ivan. That was probably overkill for my immediate needs but it is really interesting and I'll file it away for future use. I'm sure I'm going to find a use for this somewhere, so thanks for taking the time to post. – Tim Long Feb 18 '16 at 06:29
1

The problem is that Entity Framework doesn't know how to translate that to SQL. Instead of filtering after you get into the database, you can just make your query include the filter as you would if writing straight SQL. It will be a bit more verbose, but it will be FAR less expensive to drag lots of data that will be immediately filtered once it gets built in memory.

What you need to do is to compare each of your times to the high and low that you're looking for.

// I prefer to move these outside the query for clarity.
var raPlus = ra.Value.AddMinute(1);
var raMinus = ra.Value.AddMinute(-1);
var decPlus = dec.Value.AddMinute(1);
var decMinus = dec.Value.AddMinute(-1);

var targets = from item in db.Targets
              where item.RightAscension <= raPlus &&
                    item.RightAscension >= raMinus &&
                    item.Declination <= decPlus &&
                    item.Declination >= decMinus
              select item;
krillgar
  • 12,596
  • 6
  • 50
  • 86
  • Accepting this as the answer - but I'm posting my own answer to show exactly how I eventually did it. – Tim Long Feb 16 '16 at 13:22
  • Ah, sorry for not thinking enough about the problem to realize that you were referring to angular "hours" and "minutes" and not time. But the principle still holds, as you showcased in your answer. – krillgar Feb 16 '16 at 14:09
0

LINQ (to EF) query provider doesn't know how to execute your IsCloseTo method in SQL. You need to enumerate your items first and than filter it by using you extension method, something like this:

var db = CompositionRoot.GetTargetDatabase();
var targets = from item in db.Targets
              select item;
//now targets will be enumarated and can be querable with LINQ to objects
var filteredTargets = from target in targets.ToList() 
               where target.RightAscension.IsCloseTo(ra.Value, WithinOneMinute) 
               && target.Declination.IsCloseTo(dec.Value, WithinOneMinute)
              select target; 
var filteredTargets = targets.ToList();
danteus
  • 76
  • 2
  • 6
  • That could be **very** expensive if there are a lot of records that would get filtered out. – krillgar Feb 16 '16 at 12:14
  • The only option is to avoid using of the extension method. – danteus Feb 16 '16 at 12:25
  • Currently there are about 100 records but that is going to increase a lot, so bringing in all rows is really not an option. However I have solved the problem another way, I will post my own answer. – Tim Long Feb 16 '16 at 13:21
0

This is how I eventually did it:

        const double radius = 1.0;        
        const double radiusHours = radius / 15.0;
        var db = CompositionRoot.GetTargetDatabase();
        var ra = rightAscension.Value;      
        var dec = declination.Value;        
        var minRa = ra - radiusHours;
        var maxRa = ra + radiusHours;
        var minDec = dec - radius;
        var maxDec = dec + radius;
        var closeTargets = from target in db.Targets
                           where target.RightAscension >= minRa 
                                 && target.RightAscension <= maxRa
                                 && target.Declination >= minDec 
                                 && target.Declination <= maxDec
                           let deltaRa = Abs(target.RightAscension - ra) * 15.0 // in degrees
                           let deltaDec = Abs(target.Declination - dec)
                           let distanceSquared = deltaRa * deltaRa + deltaDec * deltaDec
                           orderby distanceSquared
                           select target;

One slight complication is that Right Ascension is in hours (15 degrees per hour) whereas Declination is in degrees, so I have to adjust for that in a couple of places.

I first narrow down the list to objects within a small radius (1 degree in this case). In fact I am using a 1 degree square, but it is a good enough approximation. Then I order the items by distance from the required point, using Pythagoras (I don't take the square root because that gives an error again, but again it is good enough to simply get the right ordering).

Then finally I materialise the query and take the first element as my answer.

This still isn't perfect because it doesn't handle the case where Right Ascension is close to zero. I will end up comparing with a negative RA instead of something near 23:59 - but I can live with it for now.

The results power a speech synthesizer that "announces" where the telescope is pointing as the name of the object. Pretty cool :) If I occasionally miss one, it doesn't really matter.

Tim Long
  • 13,508
  • 19
  • 79
  • 147