remove duplicated data in list C#?

Question

I have list with duplicated values, what I'm trying to do is to remove the data that is duplicate. If you look at the list data you will see that 'proj5,9,2013,FC,5558', for example, is duplicate. update p is a ReportByProjectModel

class ReportByProjectModel
{
        public string projectID { get; set; }
        public int month { get; set; }
        public int year { get; set; }
        public string type { get; set; }
}
List<ReportByProjectModel> uniqueProjectList = p.Distinct().ToList();

System.Diagnostics.Trace.WriteLine("data list Distinct");

foreach (var d in uniqueProjectList.OrderBy(t => t.projectID).Distinct())
{
    System.Diagnostics.Trace.WriteLine(d.projectID + "," + d.month + "," + d.year + "," + d.type + "," + d.value);
}

The list data:

Proj1,4,2013,FC,1001
Proj1,5,2013,FC,664
Proj1,5,2014,FC,5850
Proj1,6,2013,FC,6651
Proj1,7,2013,FC,1785
Proj1,8,2013,FC,5907
Proj1,9,2013,FC,36582
Proj1,10,2013,FC,57170
Proj1,11,2013,FC,46002
Proj1,12,2013,FC,36758
proj2,10,2013,FC,291
proj2,10,2013,FC,291
Proj3,10,2013,FC,1440
Proj3,11,2013,FC,4105
Proj3,12,2013,FC,3223
Proj3,10,2013,FC,1440
Proj3,11,2013,FC,4105
Proj3,12,2013,FC,3223
Proj4,10,2013,FC,646
Proj4,11,2013,FC,1400
Proj4,12,2013,FC,350
Proj4,10,2013,FC,646
Proj4,11,2013,FC,1400
Proj4,12,2013,FC,350
proj5,9,2013,FC,5558
proj5,10,2013,FC,31168
proj5,11,2013,FC,29807
proj5,12,2013,FC,23329
proj5,9,2013,FC,5558
proj5,10,2013,FC,31168
proj5,11,2013,FC,29807
proj5,12,2013,FC,23329
proj6,10,2013,FC,13455
proj6,11,2013,FC,24313
proj6,12,2013,FC,10116
proj6,10,2013,FC,13455
proj6,11,2013,FC,24313
proj6,12,2013,FC,10116
Proj7,11,2013,FC,12834
Proj7,12,2013,FC,697
proj8,1,2014,FC,20000
proj8,2,2014,FC,4250
proj8,3,2014,FC,1250
proj8,4,2014,FC,2398
proj8,11,2013,FC,7339
proj8,12,2013,FC,13038

view this thread http://stackoverflow.com/questions/10255121/get-list-of-distinct-values-in-listfoo — Sorceri, Sep 05 '14 at 14:00

score 3 · Answer 1 · answered Sep 05 '14 at 13:59

3

Implement IEquatable<T> on ReportByProjectModel and Distinct() should function as you want it to.

The default equality comparer, Default, is used to compare values of the types that implement the IEquatable generic interface.

More info: Enumerable.Distinct()

answered Sep 05 '14 at 13:59

toadflakz

7,764
1
27
40

Alternatively you could use a [DistinctBy](http://stackoverflow.com/questions/1300088/distinct-with-lambda/1300116#1300116) method. – Casey Sep 05 '14 at 14:02
1

You could also perform a GroupBy on all columns to eliminate the duplicates – Terence Sep 05 '14 at 14:08
@Terence it would be nice to see that solution too – Liath Sep 05 '14 at 14:09

Liath · Accepted Answer · 2014-09-05T14:08:08.863

Distinct() uses the .Equals() and .GetHashCode() methods of the objects you're sorting.

The base implementation of this is ReferenceEquals(). I would suggest overriding these methods in your ReportByProjectModel class.

If you do this when your .Distinct() is called whether or not the entities are the same will be based on their properties rather than their references and duplicates will be filtered out.

public class ReportByProjectModel
{
        public string projectID { get; set; }
        public int month { get; set; }
        public int year { get; set; }
        public string type { get; set; }

    public override bool Equals(System.Object obj)
    {
        if (obj == null)
        {
            return false;
        }

        // If parameter is the wrong type then return false.
        ReportByProjectModel p = obj as TwoDPoint;
        if (p == null)
        {
            return false;
        }

        // Return true if the fields match:
        return obj.projectID  == p.projectID
           && obj.month == p.month 
           && obj.year == p.year
           && obj.type == p.type;
    }

    public override int GetHashCode()
    {
        return string.Concat(projectID, "|", month, "|", year "|", type).GetHashCode();
    }
}

I'm not 100% about my implementation of GetHashCode, the idea is to create a unique key for this object (based on the values which make it unique). As you know your solution you could probably come up with a better mechanism.

score -1 · Answer 3 · answered Sep 05 '14 at 14:00

-1

Use distinct.

uniquePojectList = uniqueProjectList.Distinct();

http://msdn.microsoft.com/en-us/library/vstudio/bb348436%28v=vs.100%29.aspx

answered Sep 05 '14 at 14:00

DidIReallyWriteThat

1,033
1
10
39

The OP is using Distinct already. – Casey Sep 05 '14 at 14:01
1

Distinct uses the default equality comparison which will not work as references are not equal... – thekip Sep 05 '14 at 14:01
not correctly, I didn't read through his entire code, but his initial linq query uses distinct on one field. So by using distinct on his current list to remove exact matches, it works correcly, just tested it... – DidIReallyWriteThat Sep 05 '14 at 14:02

remove duplicated data in list C#?

3 Answers3