Remove Duplicate Objects from list in VB.Net

Question

I am trying to remove duplicate object from a list based on the companyID.

How do I integrate through a list and remove the object based on a companyID.

While reader.Read()
            companys.Add(New CompanySearch)
            companys(companys.Count - 1).StartDate = reader("StartDate").ToString & " (" & count & ")"
            companys(companys.Count - 1).CompanyID = reader("company").ToString
            companys(companys.Count - 1).Origin = reader("Origin").ToString
            companys(companys.Count - 1).OriginName = reader("OriginName").ToString
            companys(companys.Count - 1).Status = reader("status").ToString
            companys(companys.Count - 1).StatusName = reader("statusname").ToString
            companys(companys.Count - 1).Status = reader("status").ToString
            companys(companys.Count - 1).FullLegalBusinessName = reader("fullLegalBusinessName")
            companys(companys.Count - 1).AmountRequestedText = reader("amountRequestedText")
            companys(companys.Count - 1).HowSoonNeededText = reader("howSoonNeededText")
            companys(companys.Count - 1).QueueID = reader("QueueID")
            companys(companys.Count - 1).Company = reader("Company")
        End While

For counter As Integer = 0 To companys.Count
        counter += 1
        If i <> CInt(companys(companys.Count - 1).CompanyID) Then
            i = CInt(companys(companys.Count - 1).CompanyID)
        Else
            companys.Remove()
        End If
    Next

Why not just prevent it from being added instead of removing it? — A Friend, Jun 14 '16 at 15:54
Because it comes from a stored procedure and as far as SQL is concerned the rows it returns are distinct — R Sharp, Jun 14 '16 at 16:03
Check [this SO question](http://stackoverflow.com/questions/2322200/remove-duplicates-from-a-listof-t-in-vb-net). Globaly, try to set a function/class that you can pass as a parameter to `.Distinct` so that you can call it and get the duplicates removed your result could be like `companys= companys.Distinct(New yourComparasionFunction).ToList` Another option (slower) is to iterate with two loops - the 1st goes through your list and the 2nd does the same inside the 1st and then you compare both companyId's — M.M, Jun 14 '16 at 16:20
it will always comeback distinct, because although the companyids are the same the origin is not always — R Sharp, Jun 14 '16 at 16:24

Joel Coehoorn · Accepted Answer · 2016-06-14T21:16:04.533

Don't add them in the first place. Use either aDictionary (if you will look them up by ID later) or a HashSet (if you won't) to check before adding to the results. Here's the HashSet example:

Dim companyIDs As New HashSet(Of String)()  
While reader.Read()
    If Not companyIDs.Contains(reader("company").ToString()) Then
        companys.Add(New CompanySearch() With {
            .StartDate = reader("StartDate").ToString() & " (" & count & ")",
            .CompanyID = reader("company").ToString(),
            .Origin = reader("Origin").ToString(),
            .OriginName = reader("OriginName").ToString(),
            .Status = reader("status").ToString(),
            .StatusName = reader("statusname").ToString(),
            .Status = reader("status").ToString(),
            .FullLegalBusinessName = reader("fullLegalBusinessName"),
            .AmountRequestedText = reader("amountRequestedText"),
            .HowSoonNeededText = reader("howSoonNeededText"),
            .QueueID = reader("QueueID"),
            .Company = reader("Company"),
        })
    End If
    companyIDs.Add(reader("company").ToString())
End While

I also noticed that both the .Company and .CompanyID properties in this object are populated from the company column in the reader. Is this intentional, or do you mean to look at a different column for .CompanyID?

Additionally, while I understand your existing search SQL already considers these company rows as distinct, you should probably go back to the drawing board there and rethink the SQL, so that you truly do get distinct records. Perhaps use a nested query or CTE to first find a projection of CompanyID values that match your query, and then join back to your company table to get the details for each company with an ID included in those initial results. If that's not possible, you should consider what it is that makes the rows different, because I promise you that some column IS different, and if you just cull one record or the other you're potentially showing the user bad data from the wrong row.

adding my data in a dictionnary instead of a simple list fixed everything for me. My code was adding obect, at multiple place, then i had to handle duplicate at the end. I just created a unique key combination to my dictionary. Thanks! — Sophie, Jul 28 '23 at 18:23

score 2 · Answer 2 · answered Jun 14 '16 at 16:29

2

Use this :

Dim distinctCompanys = companys.GroupBy(Function(x) x.CompanyID).Select(Function(y) y.First())

answered Jun 14 '16 at 16:29

Abdellah OUMGHAR

3,627
1
11
16

score 1 · Answer 3 · answered Jun 14 '16 at 16:35

You can easily filter the collection with LINQ:

Dim companies = companys.Distinct(Function(c) c.CompanyID).ToList

Or use Dictionary(Of String, CompanySearch) instead, for Example:

Dim companies As Dictionary(Of String, CompanySearch)

While reader.Read()
    Dim companyID = reader("company").ToString
    companies(companyID) = New CompanySearch() With {
        .StartDate = reader("StartDate").ToString & " (" & count & ")",
        .CompanyID = companyID,
        .Origin = reader("Origin").ToString,
        .OriginName = reader("OriginName").ToString,
        .Status = reader("status").ToString,
        .StatusName = reader("statusname").ToString,
        .Status = reader("status").ToString,
        .FullLegalBusinessName = reader("fullLegalBusinessName"),
        .AmountRequestedText = reader("amountRequestedText"),
        .HowSoonNeededText = reader("howSoonNeededText"),
        .QueueID = reader("QueueID"),
        .Company = reader("Company")
    }
End While

But I recommend grouping instead, so that you can check for duplicates after:

Dim companiesLookup = companys.ToLookup(Function(c) c.CompanyID)

Dim duplicates = companiesLookup.Where(Function(c) c.Count > 1).ToList

Remove Duplicate Objects from list in VB.Net

3 Answers3