vb.net fastest way to remove duplicates from listview?

Question

Does anyone have a quicker and better way to remove duplicates from a listview? I am currently doing it like this: I sort the items alphabetically, and then it checks the item below and compares it with the one above.

This is time consuming though.. When I enter 20.000 records into an excel sheet, and remove duplicates it takes a few miliseconds, but with this code below it takes hours to check 20.000 items in vb.net. Does anyone know of a faster method?

 Dim max As Integer = ListView2.Items.Count
    Dim i As Integer = 0

    For Each item As ListViewItem In ListView2.Items

        If i = max Then
            Exit For
        End If

        If i > 0 Then

            If item.Text = ListView2.Items(i - 1).Text Then
                max -= 1
                item.Remove()
                i -= 1
            End If
        End If

        i += 1

        Label4.Text = "Total domains: " & ListView2.Items.Count

    Next

It's usually a bad idea to remove items during a for-each loop. Usually it's better to work off a copy of the list or go in reverse order in a loop. The "slowness" might be attributed to the control trying to redraw everything on every item removal. See [How do I suspend painting for a control and its children?](http://stackoverflow.com/q/487661/719186) — LarsTech, Apr 14 '17 at 15:10
Similarly, use a DataGridView bound to a datasource which would allow you to easily check that source and not add dupes to begin with. In any case managing the source will be faster. Controls are not intended for data storage but to present a *view* of the data to the user. — Ňɏssa Pøngjǣrdenlarp, Apr 14 '17 at 15:37

score 1 · Answer 1 · answered Apr 14 '17 at 16:26

Here is a Linq based solution to get distinct items based on item.Text and sort the items. If sorting is not required, you can remove the OrderBy part.

Private Shared Sub RemoveDuplicatesAndSort(lv As ListView)
    Dim distictItems As ListViewItem() = lv.Items.Cast(Of ListViewItem)().Distinct(New LVItemComparer()).OrderBy(Function(item As ListViewItem) item.Text).ToArray
    lv.BeginUpdate() ' suppress screen updates
    lv.Items.Clear()
    lv.Items.AddRange(distictItems)
    lv.EndUpdate()
End Sub

Private Class LVItemComparer : Implements IEqualityComparer(Of ListViewItem)
    Public Function Equals1(x As ListViewItem, y As ListViewItem) As Boolean Implements IEqualityComparer(Of ListViewItem).Equals
        Return x.Text.Equals(y.Text)
    End Function

    Public Function GetHashCode1(obj As ListViewItem) As Integer Implements IEqualityComparer(Of ListViewItem).GetHashCode
        Return obj.Text.GetHashCode
    End Function
End Class

Fabio · Answer 2 · 2017-04-14T15:39:23.557

0

Use HashSet which will accept only unique values.

Dim itemsText = ListView2.Items.Cast(Of ListViewItem).Select(Function(item) item.Text)
Dim uniquesValues As HashSet(Of String) = New HashSet(Of String)(itemsText)

Then set items from HashSet to the ListView.

edited Apr 14 '17 at 15:39

answered Apr 14 '17 at 15:30

Fabio

31,528
4
33
72

I get 2 errors with this code. 1) 'Select' is not a member of 'System.Windows.Forms.ListView.ListViewItemCollection'. and 2) Overload resolution failed because no accessible 'New' can be called without a narrowing conversion: – Anders Apr 14 '17 at 15:38
@Anders - you need to cast items to `ListViewItem` before using `Select` - check updated answer – Fabio Apr 14 '17 at 15:40
So it will store the new items to uniquesValues? I have been testing it, but doesn't really seem to work. – Anders Apr 14 '17 at 15:48

vb.net fastest way to remove duplicates from listview?

2 Answers2