Threads returning different values with static data

Question

I am making a tool that runs on a large directory extracting data and starts a thread per language code (the first level of files in the directory). I added a loop that stops the threads from adding to the database until all threads are finished because the tool was going into deadlock without it. However, when testing this data, the DB is storing the wrong amount of languages even though the test data is static. For example I have 67 languages, but my DB only has 48 in it. I think the issue may be that my loop for stopping the program from proceeding before the threads have stopped may be broken, ie. it is adding files to the DB before all threads have stopped, thus losing languages along the way. I don't suppose anyone has come across a similar issue or knows of a way to solve this problem? Thanks.

 //get the languages from the folders
        string[] filePaths = Directory.GetDirectories(rootDirectory);
        for (int i = 0; i < filePaths.Length; i++)
        {
            string LCID = filePaths[i].Split('\\').Last();
            Console.WriteLine(LCID);
            //go through files in each folder and sub-folder with threads
            Thread t1 = new Thread(() => new HBScanner(new DirectoryInfo(filePaths[i - 1])).HBscan());
            t1.Start();
            threads.Add(t1);
        }

        // wait for all threads to complete before proceeding
        foreach (Thread thread in threads)
        {
            while (thread.ThreadState != ThreadState.Stopped)
            {
                //wait
            }
        }

You might want to consider using the `Task` instead of individual threads, this kind of construct could easily get out of control and have you spinning off hundreds or thousands of threads. Also, if you're going to loop and wait, you should use a `WaitHandle` of some sort, or you might want to check the `ThreadState == ThreadState.Running`. There are a lot of other states a thread might enter instead of Stopped. — CodingGorilla, Feb 05 '13 at 15:27
when I don't have filepaths[I-1] and just have filepaths[I] it gives an error "index outside the bounds of array" and when I looked it up online the solution seemed to be [I-1] — RebeccaD, Feb 05 '13 at 15:42

score 2 · Accepted Answer · edited May 23 '17 at 10:32

2

First and foremost: make a local copy of the path and pass that into the thread instead of the for-loop variable. Closing over the loop variable is considered harmful.

I don't know why you get an index out of range exception, but you can also avoid that by using a foreach-loop.

//get the languages from the folders
string[] filePaths = Directory.GetDirectories(rootDirectory);
foreach(string filePath in filePaths)
{
    Console.WriteLine(filePath.Split('\\').Last());

    string tmpPath = filePath; // <-- local copy

    //go through files in each folder and sub-folder with threads
    Thread t1 = new Thread(() => new HBScanner(new DirectoryInfo(tmpPath)).HBscan());
    t1.Start();
    threads.Add(t1);
}

Second: use Join on the threads instead of having some custom wait code.

// wait for all threads to complete before proceeding
foreach (Thread thread in threads)
{
    thread.Join();
}

Finally, make sure that there is no contention on the database. Without further information on what the HBScanner does, it's difficult to say anything else about what might be causing this issue.

edited May 23 '17 at 10:32

Community

1
1

answered Feb 05 '13 at 16:00

Kiril

39,672
31
167
226

How does this join function work? I am trying to understand it from documentation but can't see anything explaining it fully. – Simon Kiely Feb 06 '13 at 09:42
I'm not 100% sure myself by as far as I know ensures a thread has terminated. It will block indefinitely if the thread does not terminate and if the thread has already terminated when Join is called, the method returns immediately. – RebeccaD Feb 06 '13 at 09:52
@SimonKiely what RebeccaD said. :) There is another way to synchronize threads, which might be a bit more efficient, and that's by using [CountdownEvent](http://msdn.microsoft.com/en-us/library/system.threading.countdownevent.aspx): the main thread waits on the countdown event, while each thread signals on the vent when it's done with the work. When all of the threads have signaled, the main thread continues. – Kiril Feb 06 '13 at 16:00

Threads returning different values with static data

1 Answers1