Is there a way to return back to a main menu loop after all tasks in a thread are completed in Python 3?

Question

I'm making a small program that can do some small tasks for me with multiple threads. Currently, I have a small menu loop that asks me what task I want to run, it then asks me for the information it needs to complete the task, and sets up a specified number of threads to complete the task through a queue.

Currently, my menu loop code looks like this and is run when the program starts:

# menu loop
def menu_loop():
    while True:
        choice = menu()
        if choice is not None:
            break
    # configure program
    if choice == 1:
        while True:
            config_choice = configure() # another menu loop
            if config_choice is not None:
                if config_choice == 0:
                    clear_screen()
                    print(f"[*] {red}exiting config...\n")
                    menu_loop()
                elif config_choice == 1:
                    num_of_threads = int(input("number of threads: "))
                    clear_screen()
                    print(f"[*] {green}number of threads set to {num_of_threads}.\n")
                    menu_loop()
                elif config_choice == 2:
                    folder_path = input("folder path for results: ")
                    clear_screen()
                    print(f"[*] {green}path to results set to {folder_path}.\n")
                    menu_loop()
                break

    if choice == 2:
        strings = [line.strip() for line in open((input("path to string file: ")), "r")] # load strings
        for string in strings:
            q.put(string) # insert each string into queue
        print(f"{green}loaded {len(strings)} strings into the checker.") # log
        with open(string_results_ml, "a+") as sr_multi_line:
            sr_multi_line.write("------------------------------\n") # first line of the file
        for i in range(int(input("number of threads: "))):
            Thread(target=check_string, args=(q,)).start() # start threads

The code for the check_string function is as follows:

def check_string(q):
    while True:
        try:
            work = q.get()
            if q.empty(): # needed because except doesn't trigger half the time
                sleep(1)
                quit()
        except queue.Empty:
            quit()
        headers = {"Content-Type": "application/json"}
        url = "https://[redacted]/{}".format(work)
        try:
            r = requests.get(url, headers=headers)
            jd = r.json()
            if r.status_code == 200:
                try:
                    result = jd["result"]
                    # write working strings and their data to the results file
                    with open(string_results, "a+") as s_r:
                        s_r.write(f"{string} - {result}\n")
                    with lock:
                        print(f"[{blue}{r.status_code}{rs}]{green} got result for {string}")
                    sleep(0.3) # sleep for a bit, time out
                except:
                    print(f"[{blue}{r.status_code}{rs}]{red} something happened while parsing the data.")
            else:
                with lock:
                    print(f"[{blue}{r.status_code}{rs}]{red} no results for {string}")
        except:
            with lock:
                print(f"{red}fatal error while checking {string}! the string has been written to a separate file for you to check later.")
            with open(fatal, "a+") as f_e:
                f_e.write(string + "\n")
        q.task_done()

Right now, what happens after all the strings are done being checked is that the program just hangs and doesn't go back to the menu loop. There is no code for it to go back to the loop currently but it should at least quit after all strings are done being checked. It just hangs currently. When I press CTRL-C, it gives me the following exception:

^CException ignored in: <module 'threading' from '/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py'>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 1307, in _shutdown
    lock.acquire()
KeyboardInterrupt

How can I make the program go back to the main menu loop after all strings are done being checked instead of just hanging?

Answer 1

Your threads are stopping because the queue is empty and they are waiting for new item to be queued. By default, queue.get() blocks if the queue is empty. In your code, check the queue before q.get(). In addition, change q.get() to q.get(false) to prevent blocking (an error will be thrown if empty queue).

while True:
    try:
        if q.empty(): quit()  # exit thread if no items left - add this line
        work = q.get(false) # don't block to be safe - update this line
        if q.empty(): # needed because except doesn't trigger half the time
            time.sleep(1)
            quit()
    except queue.Empty:
        quit()