How to I get this LINQ full-outer-join to function properly?

Question

I am building a WPF application that monitors a directory on the users computer. The app uploads files from the monitored directory and then saves off some information into a SQLite db. Part of the business processing is to not re-process files that have already been uploaded and to re-upload files that have been uploaded but have changed since the last upload.

I have two helper methods that build and return a List<FileMetaData> that I used LINQ - Full Outer Join to join the. My problem is that the code doesn't seem to work when I use my FileMetaData object. It seems like everything should all work but I'm at a loss as to why it's not working. I would normally try to post as a comment on the other thread but I don't currently have the "Rep" here to do that.

Below is a sample I've built that shows my problem if you run it in LINQpad. Make sure you set the language as "C# Program" before you click the run button. What should I do differently to have the sample work with the objects? Thanks a ton!

    void Main()
    {
        var dbItems = new List<FileMetaData>() { 
                new FileMetaData {FilePath = "C:\\Foo.txt", DbTimestamp = "1" },
                new FileMetaData {FilePath = "C:\\FooBar.txt", DbTimestamp = "3" },
            };

        var fsItems = new List<FileMetaData>() {
                new FileMetaData {FilePath = "C:\\Bar.txt", FsTimestamp = "2" },
                new FileMetaData {FilePath = "C:\\FooBar.txt", FsTimestamp = "3" },
            };

            var leftOuter = from d in dbItems
                    join f in fsItems on d.FilePath equals f.FilePath
                    into temp
                    from o in temp.DefaultIfEmpty(new FileMetaData(){})
                    select new FileMetaData { 
                        FilePath = d.FilePath, 
                        DbTimestamp = d.DbTimestamp,
                        FsTimestamp = o.FsTimestamp,
                    };

            var rightOuter = from f in fsItems
                    join d in dbItems on f.FilePath equals d.FilePath
                    into temp
                    from o in temp.DefaultIfEmpty(new FileMetaData(){})
                    select new FileMetaData { 
                        FilePath = f.FilePath, 
                        DbTimestamp = o.DbTimestamp,
                        FsTimestamp = f.FsTimestamp,
                    };

            var full = leftOuter.AsEnumerable().Union(rightOuter.AsEnumerable());

            leftOuter.Dump("Left Results");
            rightOuter.Dump("Right Results");

            full.Dump("Full Results");
    }

    // Define other methods and classes here
    public class FileMetaData
    {
        public string FilePath;
        public string DbTimestamp;
        public string FsTimestamp;
    }

EDIT:

The answer below was exactly what I was looking for. I implemented the IEqualityComparer as defined below and changed my call to var full = leftOuter.Union(rightOuter, new FileMetaDataCompare())...

    public class FileMetaDataCompare : IEqualityComparer<FileMetaData>
    {
        public bool Equals(FileMetaData x, FileMetaData y)
        {
            var areEqual = x.FilePath == y.FilePath;
            areEqual = areEqual && x.DbTimestamp == y.DbTimestamp;
            areEqual = areEqual && x.FsTimestamp == y.FsTimestamp;

            return areEqual;
        }

        public int GetHashCode(FileMetaData obj)
        {
            var hCode = string.Concat(obj.FilePath, obj.DbTimestamp, obj.FsTimestamp);
            return hCode.GetHashCode();
        }
    }

score 4 · Accepted Answer · answered Jul 24 '12 at 17:40

The problem is that Union will give you the results eliminating duplicates by checking for equality. When you use anonymous types, the definition of equality is 'all fields have an equal value'. when you declare a type, it will use the Equals method. Since you have not overridden Equals, it defaults to ReferenceEquals, which means that two separate instances are not equal regardless of their field values.

Three ways to solve this:

1) Use anonymous types in your queries and convert to defined types after Union:

var full = leftOuter.Union(rightOuter).Select(
    i=> new FileMetaData {
        FilePath = i.FilePath,
        DbTimestamp = i.DbTimestamp,
        FsTimestamp = i.FsTimestamp
    });

2) Define an IEqualityComparer<FileMetaData> class that defines equality thay you want to (just FilePath? all fields?) and pass an instance of it to Union()

3) Override Equals() (and GetHashCode()) in FileMetaData.

2) and 3) will be very similar, but overriding Equals() can (and will) be used whenever you check for equality, not just in this situation.

Thanks a ton for the answer. I completely blanked on IEqualityComparer. — Frito, Jul 24 '12 at 18:38

How to I get this LINQ full-outer-join to function properly?

1 Answers1

Linked