Calculate difference from previous item with LINQ

Question

I'm trying to prepare data for a graph using LINQ.

The problem that i cant solve is how to calculate the "difference to previous.

the result I expect is

ID= 1, Date= Now, DiffToPrev= 0;

ID= 1, Date= Now+1, DiffToPrev= 3;

ID= 1, Date= Now+2, DiffToPrev= 7;

ID= 1, Date= Now+3, DiffToPrev= -6;

etc...

Can You help me create such a query ?

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace ConsoleApplication1
{
    public class MyObject
    {
        public int ID { get; set; }
        public DateTime Date { get; set; }
        public int Value { get; set; }
    }

    class Program
    {
        static void Main()
        {
               var list = new List<MyObject>
          {
            new MyObject {ID= 1,Date = DateTime.Now,Value = 5},
            new MyObject {ID= 1,Date = DateTime.Now.AddDays(1),Value = 8},
            new MyObject {ID= 1,Date = DateTime.Now.AddDays(2),Value = 15},
            new MyObject {ID= 1,Date = DateTime.Now.AddDays(3),Value = 9},
            new MyObject {ID= 1,Date = DateTime.Now.AddDays(4),Value = 12},
            new MyObject {ID= 1,Date = DateTime.Now.AddDays(5),Value = 25},
            new MyObject {ID= 2,Date = DateTime.Now,Value = 10},
            new MyObject {ID= 2,Date = DateTime.Now.AddDays(1),Value = 7},
            new MyObject {ID= 2,Date = DateTime.Now.AddDays(2),Value = 19},
            new MyObject {ID= 2,Date = DateTime.Now.AddDays(3),Value = 12},
            new MyObject {ID= 2,Date = DateTime.Now.AddDays(4),Value = 15},
            new MyObject {ID= 2,Date = DateTime.Now.AddDays(5),Value = 18}

        };

            Console.WriteLine(list);   

            Console.ReadLine();
        }
    }
}

Answer 1

One option (for LINQ to Objects) would be to create your own LINQ operator:

// I don't like this name :(
public static IEnumerable<TResult> SelectWithPrevious<TSource, TResult>
    (this IEnumerable<TSource> source,
     Func<TSource, TSource, TResult> projection)
{
    using (var iterator = source.GetEnumerator())
    {
        if (!iterator.MoveNext())
        {
             yield break;
        }
        TSource previous = iterator.Current;
        while (iterator.MoveNext())
        {
            yield return projection(previous, iterator.Current);
            previous = iterator.Current;
        }
    }
}

This enables you to perform your projection using only a single pass of the source sequence, which is always a bonus (imagine running it over a large log file).

Note that it will project a sequence of length n into a sequence of length n-1 - you may want to prepend a "dummy" first element, for example. (Or change the method to include one.)

Here's an example of how you'd use it:

var query = list.SelectWithPrevious((prev, cur) =>
     new { ID = cur.ID, Date = cur.Date, DateDiff = (cur.Date - prev.Date).Days) });

Note that this will include the final result of one ID with the first result of the next ID... you may wish to group your sequence by ID first.

Answer 2

Use index to get previous object:

   var LinqList = list.Select( 
       (myObject, index) => 
          new { 
            ID = myObject.ID, 
            Date = myObject.Date, 
            Value = myObject.Value, 
            DiffToPrev = (index > 0 ? myObject.Value - list[index - 1].Value : 0)
          }
   );

Answer 3

In C#4 you can use the Zip method in order to process two items at a time. Like this:

        var list1 = list.Take(list.Count() - 1);
        var list2 = list.Skip(1);
        var diff = list1.Zip(list2, (item1, item2) => ...);

Answer 4

Modification of Jon Skeet's answer to not skip the first item:

public static IEnumerable<TResult> SelectWithPrev<TSource, TResult>
    (this IEnumerable<TSource> source, 
    Func<TSource, TSource, bool, TResult> projection)
{
    using (var iterator = source.GetEnumerator())
    {
        var isfirst = true;
        var previous = default(TSource);
        while (iterator.MoveNext())
        {
            yield return projection(iterator.Current, previous, isfirst);
            isfirst = false;
            previous = iterator.Current;
        }
    }
}

A few key differences... passes a third bool parameter to indicate if it is the first element of the enumerable. I also switched the order of the current/previous parameters.

Here's the matching example:

var query = list.SelectWithPrevious((cur, prev, isfirst) =>
    new { 
        ID = cur.ID, 
        Date = cur.Date, 
        DateDiff = (isfirst ? cur.Date : cur.Date - prev.Date).Days);
    });

Answer 5

Further to Felix Ungman's post above, below is an example of how you can achieve the data you need making use of Zip():

        var diffs = list.Skip(1).Zip(list,
            (curr, prev) => new { CurrentID = curr.ID, PreviousID = prev.ID, CurrDate = curr.Date, PrevDate = prev.Date, DiffToPrev = curr.Date.Day - prev.Date.Day })
            .ToList();

        diffs.ForEach(fe => Console.WriteLine(string.Format("Current ID: {0}, Previous ID: {1} Current Date: {2}, Previous Date: {3} Diff: {4}",
            fe.CurrentID, fe.PreviousID, fe.CurrDate, fe.PrevDate, fe.DiffToPrev)));

Basically, you are zipping two versions of the same list but the first version (the current list) begins at the 2nd element in the collection, otherwise a difference would always differ the same element, giving a difference of zero.

I hope this makes sense,

Dave

Answer 6

Yet another mod on Jon Skeet's version (thanks for your solution +1). Except this is returning an enumerable of tuples.

public static IEnumerable<Tuple<T, T>> Intermediate<T>(this IEnumerable<T> source)
{
    using (var iterator = source.GetEnumerator())
    {
        if (!iterator.MoveNext())
        {
            yield break;
        }
        T previous = iterator.Current;
        while (iterator.MoveNext())
        {
            yield return new Tuple<T, T>(previous, iterator.Current);
            previous = iterator.Current;
        }
    }
}

This is NOT returning the first because it's about returning the intermediate between items.

use it like:

public class MyObject
{
    public int ID { get; set; }
    public DateTime Date { get; set; }
    public int Value { get; set; }
}

var myObjectList = new List<MyObject>();

// don't forget to order on `Date`

foreach(var deltaItem in myObjectList.Intermediate())
{
    var delta = deltaItem.Second.Offset - deltaItem.First.Offset;
    // ..
}

OR

var newList = myObjectList.Intermediate().Select(item => item.Second.Date - item.First.Date);

OR (like jon shows)

var newList = myObjectList.Intermediate().Select(item => new 
{ 
    ID = item.Second.ID, 
    Date = item.Second.Date, 
    DateDiff = (item.Second.Date - item.First.Date).Days
});

Answer 7

Here is the refactored code with C# 7.2 using the readonly struct and the ValueTuple (also struct).

I use Zip() to create (CurrentID, PreviousID, CurrDate, PrevDate, DiffToPrev) tuple of 5 members. It is easily iterated with foreach:

foreach(var (CurrentID, PreviousID, CurrDate, PrevDate, DiffToPrev) in diffs)

The full code:

public readonly struct S
{
    public int ID { get; }
    public DateTime Date { get; }
    public int Value { get; }

    public S(S other) => this = other;

    public S(int id, DateTime date, int value)
    {
        ID = id;
        Date = date;
        Value = value;
    }

    public static void DumpDiffs(IEnumerable<S> list)
    {
        // Zip (or compare) list with offset 1 - Skip(1) - vs the original list
        // this way the items compared are i[j+1] vs i[j]
        // Note: the resulting enumeration will include list.Count-1 items
        var diffs = list.Skip(1)
                        .Zip(list, (curr, prev) => 
                                    (CurrentID: curr.ID, PreviousID: prev.ID, 
                                    CurrDate: curr.Date, PrevDate: prev.Date, 
                                    DiffToPrev: curr.Date.Day - prev.Date.Day));

        foreach(var (CurrentID, PreviousID, CurrDate, PrevDate, DiffToPrev) in diffs)
            Console.WriteLine($"Current ID: {CurrentID}, Previous ID: {PreviousID} " +
                              $"Current Date: {CurrDate}, Previous Date: {PrevDate} " +
                              $"Diff: {DiffToPrev}");
    }
}

Unit test output:

// the list:

// ID   Date
// ---------------
// 233  17-Feb-19
// 122  31-Mar-19
// 412  03-Mar-19
// 340  05-May-19
// 920  15-May-19

// CurrentID PreviousID CurrentDate PreviousDate Diff (days)
// ---------------------------------------------------------
//    122       233     31-Mar-19   17-Feb-19      14
//    412       122     03-Mar-19   31-Mar-19      -28
//    340       412     05-May-19   03-Mar-19      2
//    920       340     15-May-19   05-May-19      10

Note: the struct (especially readonly) performance is much better than that of a class.

Thanks @FelixUngman and @DavidHuxtable for their Zip() ideas!