1

I have a scenario where I get some data from two sources (over the web). The data comes in chunks and are continuously arriving until I end my program. The data from the two sources are stored in two List<T>'s. T is different for the two lists.

I then Join the two lists on some parameter into a third type, and after joining then, I would like to look in each lists for data that was not possible to join.

The sequence is:

  1. Accumulate data in lists
  2. Join lists
  3. Clear lists
  4. Put unjoinable data in lists
  5. Repeat

So far I have a pretty nice linq query centered around a Join, and it works nicely. But when I come to checking if the two lists contain data that was not able to be joined, I have a linq query which essentially iterates through all three lists looking. This is a bit silly and kind of slow, so I was wondering if there is a nice way of joining data in linq while also creating a list of unjoinable data from the two lists.

kasperhj
  • 10,052
  • 21
  • 63
  • 106
  • 1
    Sounds like you need outer joins - similar question: http://stackoverflow.com/questions/5489987/linq-full-outer-join – davisoa Feb 20 '14 at 20:12
  • @davisoa The problem here, is that I need to filter out the joinable data and put in a list anyway. And then I would need to create two lists of the remaining full outer join to separate the unjoinable values into each of the corresponding lists. – kasperhj Feb 20 '14 at 20:25
  • 1
    I would put it all in a joined list, and then use `GroupBy` to pull out the successful joins from the unjoinable values. – davisoa Feb 20 '14 at 20:28
  • FullOuterJoin saved the day! – kasperhj Feb 21 '14 at 12:10

1 Answers1

1

Looks like you'd want to use Except

pwnyexpress
  • 1,016
  • 7
  • 14
  • I agree. I'd also take a look at [intersect](http://msdn.microsoft.com/en-us/library/system.linq.enumerable.intersect(v=vs.110).aspx) and [zip](http://msdn.microsoft.com/en-us/library/dd267698(v=vs.110).aspx) Or even [RX extensions](http://msdn.microsoft.com/en-us/data/gg577611.aspx) – nportelli Feb 20 '14 at 20:24
  • Perhaps I wasn't clear enough (I've now edited the q). The three lists are of different types. – kasperhj Feb 20 '14 at 20:27
  • 2
    The lists may be of different types, but if the composite type contains unique IDs of the two constituent lists, you can do an EXCEPT on ids – maf748 Feb 20 '14 at 20:30
  • Isn't except iterating through the lists? So we end up with iteration of creating the joined lists, and the iteration for the except. – kasperhj Feb 21 '14 at 06:37
  • If you're trying to prevent loops I would suggest taking a look at [this](http://www.25hoursaday.com/weblog/2008/06/16/FunctionalProgrammingInC30HowMapReduceFilterCanRockYourWorld.aspx) article on implementing map and reduce functions in C#. Otherwise you're not programming in a functional language, so loops are going be everywhere or hidden from you by a library. – pwnyexpress Feb 21 '14 at 17:12