Multithreading duplicate error after background worker added

Question

I wrote a program that does queries on some search engines that I wanted to be multithreaded so it could do the searches faster. I made a question about this which was answered indirectly, but I have yet another problem even after I set up delegates and such.

I was running the queries on the main thread, so I had to set up a BackgroundWorker. I did that but even though everything seems to run without an issue, I get duplicates of my results even though the program is suppose to do the opposite. Sometimes it doesn't even read or output a result that I know would be good.

Imports System.Net
Imports System.IO
Imports System.ComponentModel

Public Class Form2
    Dim i As Integer
    Dim SearchString As String
    Public CleanSearchStrings As String()

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        i = RichTextBox1.Lines.Count
        BackgroundWorker1.RunWorkerAsync()
    End Sub

    Private Sub StartThreads()
        While i > 0
            i = i - 1
            Dim thread_count As String = Process.GetCurrentProcess().Threads.Count - 20
            Label_T(thread_count)
            SearchString = LineFunc(i)
            Threading.Thread.Sleep(500)
            SearchString = Ask_Query(SearchString) #This was commented out
            SearchString = Bing_Query(SearchString) #just simple webscraping
            SearchString = Yahoo_Query(SearchString) 
            If SearchString.Contains("All_Query:Yes") Then
                SearchString = SearchString.Replace("All_Query:Yes", "")
                RTB(SearchString)
            End If
        End While
    End Sub

    Private Delegate Sub UpdateStatus(ByVal s As String)
    Private Delegate Sub UpdateLabel(ByVal thread_count As String)
    Private Delegate Function Line(ByVal i As Integer)

    Function LineFunc(ByVal i As Integer)
        If Me.InvokeRequired Then
            Me.Invoke(New Line(AddressOf LineFunc), New Object() {i})
        Else
            SearchString = RichTextBox1.Lines(i).ToString
            Return SearchString
        End If
    End Function

    Sub RTB(ByVal s As String)
        If Me.InvokeRequired Then
            Me.Invoke(New UpdateStatus(AddressOf RTB), New Object() {s})
        Else
            RichTextBox2.AppendText(Environment.NewLine & s)
        End If
    End Sub

    Sub Label_T(ByVal thread_count As String)
        If Me.InvokeRequired Then
            Me.Invoke(New UpdateLabel(AddressOf Label_T), New Object() {thread_count})
        Else
            Label3.Text = "Threads Running:   " + thread_count
        End If
    End Sub

    Private Sub BackgroundWorker1_DoWork(sender As Object, e As System.ComponentModel.DoWorkEventArgs) Handles BackgroundWorker1.DoWork
        BackgroundWorker1.WorkerSupportsCancellation = True
        BackgroundWorker1.WorkerReportsProgress = True

        Dim count As Integer
        Dim num As Integer = TextBox1.Text - 1
        For count = 0 To num
            Dim thread = New Threading.Thread(AddressOf StartThreads)
            thread.IsBackground = True
            thread.Start()
            Threading.Thread.Sleep(500)
        Next
    End Sub
End Class

I know it has something to do with the LineFunc that I wrote but I can't seem to figure it out.

This was working flawlessly on multithreading, but the UI would be frozen and after I added the BackgroundWorker it seems to be giving me these duplicate results error and not reading the TextBox fully.

Update:

      Private Sub StartThreads()
    For count = count To i
        Dim SearchString As String = LineFunc(count)
        count += 1
        Dim thread_count As String = CType(Process.GetCurrentProcess().Threads.Count - 20, String)
        Label_T(thread_count)
        Threading.Thread.Sleep(500)
        SearchString = CType(Ask_Query(SearchString), String)
        SearchString = CType(Bing_Query(SearchString), String)
        SearchString = CType(Yahoo_Query(SearchString), String)
        If SearchString.Contains("All_Query:Yes") Then
            SearchString = SearchString.Replace("All_Query:Yes", "")
            RTB(SearchString)
        End If
    Next
End Sub

Heres my Revised Get Line Func

Private Delegate Function GetTextBox(ByVal index As Integer) As String

Public Function LineFunc(ByVal index As Integer) As String
    If Me.InvokeRequired Then
        Me.Invoke(New GetTextBox(AddressOf LineFunc), New Object() {index})
    Else
        Dim indexSearchString As String
        indexSearchString = CType(RichTextBox1.Lines(index), String)
        Return indexSearchString
    End If
End Function

If i use the a msgbox it gives me the right values i don't understand how come it does not give me the right values when i do the func Dim SearchString As String = LineFunc(count) it returns a null string in the end i don't understand why

this seemed to work for the duplicate issue but now i seem to be having issues with my output to richtextbox2 only outputing blanks and the line func is only giving out blanks i don't understand why either but atleast there is progress

It seems like all your threads are modifying the SearchString variable, so your results could be wildly inconsistent.. I don't see how your `i` variable works when every thread is trying to bring that value down to zero. You've had previous advice to turn Option Strict On — don't ignore it. — LarsTech, Jul 07 '16 at 18:48
i already turned it on i have not ignored it and how else am i suppose to get an unique search string into every thread? — drukoz, Jul 07 '16 at 18:57
Option Strict On will put an error on this line: `Dim num As Integer = TextBox1.Text - 1` — LarsTech, Jul 07 '16 at 18:58
so it grabs the line since its indexed how its suppose to work is start -> i = rtb index -> thread reads line - > another thread reads i -> index moved down one -> read line (which would be next line) — drukoz, Jul 07 '16 at 18:59
http://i.imgur.com/hqKx2Mn.png here you can see i have it turned on — drukoz, Jul 07 '16 at 19:02
Thread 1 can make your `i` variable go to zero, so the other threads will have nothing to do. Also, *type* `Option Strict On` at the top of your code file. — LarsTech, Jul 07 '16 at 19:02
it can but mean while it goes thought the web scraping other threads will have something to do as well as long as it reaches zero it doesn't really matter which thread does it plus i'm putting in a list of 2k-3k search strings in here turned it on i see i thought i had it on before ty — drukoz, Jul 07 '16 at 19:14
That screenshot you provided that shows `Option Strict` set to `On` is in the IDE Options. That means that any projects you create after setting that value will have `Option Strict On` by default. It has zero effect on any existing projects though. You have to set `Option Strict On` for this project in the project properties. You can add it to each code file explicitly but that is bad practice. You should set it `On` for the project and then only turn it `Off` explicitly in those files that require it, which be almost none. — jmcilhinney, Jul 08 '16 at 01:21
i already got it to work with that but that was not my issue if anyone could help i am still trying to figure it out — drukoz, Jul 08 '16 at 03:33
@jmcilhinney would you have any suggestions on how to fix my duplication issue ? — drukoz, Jul 08 '16 at 22:29
@LarsTech would you have any other idea to help me ? on my duplication — drukoz, Jul 08 '16 at 23:36
So basically, you want to iterate through each line of `RichTextBox1` and pass each line to your 3 search subs in parallel and append the results from your 3 searches into `RichTextBox2` and show the number of threads running and limit the number of threads to the number in `Textbox1`. I think there is probably a simpler way of multithreading this using `Parallel.ForEach` although using this wont allow you to control the number of threads. As @LarsTech says though, your 3 lines that initiate the searches all modify the original search string and you're going to get very unpredictable results. — David Wilson, Jul 09 '16 at 07:42

David Wilson · Answer 1 · 2016-07-09T07:59:47.920

I can't verify if this will work as you haven't posted the search functions, but here goes.

Public Class Form2
    Dim i As Integer
    Dim SearchString As String
    Dim ResultString As String
    Public CleanSearchStrings As String()
    Private threadCount As Integer

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        SearchUsingRTB1Items()
    End Sub

    Private Sub SearchUsingRTB1Items()
        threadCount = 0
        Dim rtb1Lines() As String = RichTextBox1.Lines
        Parallel.ForEach(rtb1Lines, Sub(line As String)
                                        dim ResultString As String =""
                                        threadCount += 1
                                        Label_T(i.ToString)
                                        ResultString =Ask_Query(line) 'This was commented out
                                        ResultString = ResultString & Bing_Query(line) 'just simple webscraping
                                        ResultString = ResultString & Yahoo_Query(line)
                                        If ResultString.Contains("All_Query:Yes") Then
                                            ResultString = ResultString.Replace("All_Query:Yes", "")
                                            RTB(ResultString)
                                        End If
                                        threadCount -= 1
                                        Label_T(i.ToString)
                                    End Sub)

    End Sub


    Private Delegate Sub UpdateStatus(ByVal s As String)
    Private Delegate Sub UpdateLabel(ByVal thread_count As String)

    Sub RTB(ByVal s As String)
        If Me.InvokeRequired Then
            Me.Invoke(New UpdateStatus(AddressOf RTB), New Object() {s})
        Else
            RichTextBox2.AppendText(Environment.NewLine & s)
        End If
    End Sub

    Sub Label_T(ByVal thread_count As String)
        If Me.InvokeRequired Then
            Me.Invoke(New UpdateLabel(AddressOf Label_T), New Object() {thread_count})
        Else
            Label3.Text = "Threads Running:   " & thread_count
        End If
    End Sub

End Class

Also as good practice, instead of using `+` to concatenate strings - use `&` to make sure that the compiler knows what you're trying to do rather than let it try to guess :-) — David Wilson, Jul 09 '16 at 08:02
oh thank you very much for this the issue that i have with this is that i can't have too many threads makign request the thing is that in the program i will be inputting around 30,000 or 40,000 search queries so this will over load my computer since im pretty sure that it will start a thread per line correct? — drukoz, Jul 09 '16 at 10:30
It should only create one thread per core - Have a look here - http://stackoverflow.com/questions/1114317/does-parallel-foreach-limits-the-number-of-active-threads .. Apologies for the delay replying - been very very busy with work lately :) — David Wilson, Jul 26 '16 at 11:03

Multithreading duplicate error after background worker added

1 Answers1