Best way to use LINQ to fill in gaps in my data

Question

Yesterday I posted the question below on how to use LINQ to transform an object with several levels using 'group' into a flat structure:

LINQ GroupBy on object with several levels

This was very kindly answered by Cédric Bignon. The reason i wanted this transformation was so I could populate ComponentArt's Silverlight XYChart component with the userData variable. However, I've just found out that there is a known bug with their component when displaying in a stacked bar format. If there's gaps in the data it does not display properly, so I need to ensure that all users have a value for all distinct values of Category. So in my original question, where I've put what I'd like userData to be populated with, I'd need to ensure [User = "Bob", Category = "International", Spend = 0.00] was also present in the results.

I've achieved this by using the following code after the LINQ statement you have given me:

// LINQ Statement provided by @Cedric 
var userData = from spend in allSpend
           from userDataPoint in
               (from monthSpend in spend.Spend
               from spendDetail in monthSpend.Detail
               group spendDetail by spendDetail.Description into g
               select new UserDataPoint
               {
                   User = spend.UserName,
                   Category = g.Key,
                   Spend = g.Sum(t => t.Amount)
               })
           select userDataPoint;
// END

List<UserDataPoint> userDataNoGaps = new List<UserDataPoint>();
userDataNoGaps = userData.ToList();
foreach (string strCategory in userData.Select(c => c.Category).Distinct())
{
    var existing = userData.Where(c => c.Category == strCategory).Select(c => c.User);
    userDataNoGaps.AddRange(userData.Where(c => !existing.Contains(c.User)).Select(c => new UserDataPoint() { User = c.User, Category = strCategory, Spend = 0 }));
}

But my code is pretty slow when I have over 1000 users and a few categories. Can this somehow be incorporated into the LINQ statement Cédric provided or am I better off filling in the gaps afterwards using the code above?

you would be better off to do a single `.ToList()`, so that `userData` becomes a `List<>` instead of a `IEnumerable<>`, and maybe build up reference-collections, eg for `Where(c => c.Category == strCategory)` by creating `ILookup<>`/`Dictionary<>`... Anyway, your main problem is deferred execution ... — , Jul 25 '13 at 09:28

score 1 · Accepted Answer · answered Jul 25 '13 at 09:58

You can prepare list of points for all Users/Categories with 0 values merge it with your userData with union.

var userDataList = userData.ToList();

var usersList = userDataList.Select(x => x.Uder).Distinct().ToList();
var categoriesList = userDataList.Select(x => x.Category).Distinct().ToList();   

// make list of UserDataPoint with 0 sums
var empty = (from user in users
            from category in categoriesList 
            select new UserDataPoint
                       {
                          User = user,
                          Category = category,
                          Spend = 0
                       }).ToList();     

var merged = userDataList.Union(empty)
                         .GroupBy(x => new { x.User, x.Category }) // here sum up empty points with real
                         .Select(new UserDataPoint {
                             User = group.Key.User,
                             Category = group.Key.Category,
                             Spend = group.Sum(y => y.Spend)
                         }).ToList();

Best way to use LINQ to fill in gaps in my data

1 Answers1

Linked