What exactly does Thread.join() do in python? Is this incorrect usage of Thread.join()?

Question

I've very recently started learning how to write multi-threaded programs in python and for starters, began experimenting without using queues.

In the code below, loadFunction is just a sample target function for the threads. It's supposed to wait until all of the threads provided in the wait (list) argument have finished executing (and I'm trying to achieve this using join()). Then afterwards start printing out the specified number range. This is the desired behavior.

In the main program, I created two threads which do not wait for any other threads to begin counting. Then I created a third thread, which is supposed to wait for the first two threads to finish executing before it begins counting.

However that is not happening. On testing, I find that the three threads start executing simultaneously and the third thread does not wait for the first two to finish executing like I intended.

So my question here is, what knowledge am I missing about the Thread.join() function, and what changes can I make to my code to achieve the desired result?

Code:

""" Note: Python 3.3 onwards required since daemon was added as an initializable
    property from python 3.3 onwards."""

import threading

def loadFunction(name, start, end, wait=[]):
    """ wait should be a list of threads to wait for """
    map(lambda th: th.join(), wait)
    for number in range(start, end):
        print("%s : %d" % (name, number))

if __name__ == "__main__":
    t1 = threading.Thread(target=loadFunction, args=("Thread1", 1, 101), name="Thread1" ,daemon=True)
    t2 = threading.Thread(target=loadFunction, args=("Thread2", 101, 201), name="Thread2", daemon=True)
    t3 = threading.Thread(target=loadFunction, args=("Thread3", 1000, 1101, [t1, t2]), name="Thread3", daemon=True)

    t1.start()
    t2.start()
    t3.start()

    # wait for all of the daemon processes to finish before we close the program
    t1.join()
    t2.join()
    t3.join()

    print("done!")

part of the result (in one run):

Thread1 : 1
Thread1 : 2
Thread1 : 3
Thread1 : 4
Thread1 : 5
Thread1 : 6
Thread2 : 101
Thread2 : 102
Thread2 : 103
Thread2 : 104
Thread2 : 105
Thread2 : 106
Thread2 : 107
Thread2 : 108
Thread2 : 109
Thread1 : 7
Thread1 : 8
Thread1 : 9
Thread1 : 10
Thread1 : 11
Thread3 : 1000
Thread1 : 12
Thread1 : 13
Thread3 : 1001
Thread1 : 14
Thread3 : 1002
Thread1 : 15
Thread3 : 1003
Thread1 : 16
Thread2 : 110

Here are a two things that were going through my mind when I was writing this code (as quoted from the official documentation):

join(timeout=None)

Wait until the thread terminates. This blocks the calling thread until the thread whose join() method is called terminates ...

In reference to my example code, would the "calling thread" be whichever thread called the loadFunction function? I have a slight suspicion that this isn't the case and that the process itself called the function and not the thread hence the thread won't be the one waiting but the process waits instead. If this is the case, how would I fix it? I have a feeling that queues would be involved... if this is the reason in the first place.

and:

A thread can be join()ed many times.

is what lead me to use join for the same thread twice.

P.S. I'm learning about threading for the first time in python, but this is after learning forking processes in C, so maybe I might have gotten something confused here due to this. If the two aren't related, then I apologize, I'm under the impression that the two are similar (though clearly not the same since one splits the process but the other creates threads within the process itself).

Thank you.

`map` does not execute until you start iterating over it. Use a for loop. — MaxNoe, Jun 26 '18 at 18:48
I think you're misunderstanding what `daemon` is for. The point is to simulate a detached child, so you don't have to wait on in. It doesn't make much sense to create daemon threads and then wait on them with `join`. I think it may be legal and work anyway, but I don't know that it is, and I don't see any reason to be using it here. — abarnert, Jun 26 '18 at 18:48
@abarnert - interesting, but all `daemon` does is skip waiting on the thread at termination. It makes sense to do joins in the normal case but to just abandon and exit on error in the main thread. In that case, `daemon` and `join` work well together. — tdelaney, Jun 26 '18 at 19:14
Possible duplicate: https://stackoverflow.com/questions/15085348/what-is-the-use-of-join-in-python-threading — katronai, Jul 17 '21 at 15:18
Does this answer your question? [What is the use of join() in Python threading?](https://stackoverflow.com/questions/15085348/what-is-the-use-of-join-in-python-threading) — katronai, Jul 17 '21 at 15:18

score 1 · Accepted Answer · answered Jun 26 '18 at 18:54

would the "calling thread" be whichever thread called the loadFunction function?

The calling thread is whichever thread called join. so t1.join() and t2.join() and t3.join() cause the main thread to block, and the join inside loadFunction would cause t3 to block, if map was not lazily evaluated.

how would I fix it?

Your joins inside loadFunction aren't executing because map does not execute any code until you iterate over it. As MaxNoe suggests, you should use an ordinary for loop instead.

def loadFunction(name, start, end, wait=[]):
    """ wait should be a list of threads to wait for """
    for t in wait:
        t.join()
    for number in range(start, end):
        print("%s : %d" % (name, number))

This worked. I'm kinda embarrassed that I messed up because of using map improperly, I didn't know that it was lazily evaluated; I'll have to read more about it. Thank you. — Hypro999, Jun 27 '18 at 04:10

score 0 · Answer 2 · answered Jun 26 '18 at 18:54

0

if you want to wait for the two first threads to finish before starting the third, don't call start() on the third thread before join()ing the other two:

t1.start()
t2.start()

# wait for the threads above to finish before starting the third
t1.join()
t2.join()

t3.start() #now start it
t3.join()  # wait for it to finish

answered Jun 26 '18 at 18:54

nosklo

217,122
57
293
297

But then the main thread is stuck and operations aren't going on the the background while main continues with other useful work. I don't see any problem with one thread depending on others to finish work. – tdelaney Jun 26 '18 at 19:18

What exactly does Thread.join() do in python? Is this incorrect usage of Thread.join()?

2 Answers2