0

i would like to understand how to use python threading and queue. my goal is to have 40 threads always alive, this is my code:

for iteration iterations: # main iteration
    dance = 1

    if threads >= len(MAX_VALUE_ITERATION) :
        threads = len(MAX_VALUE_ITERATION)-1 # adjust number of threads as because in this iteration i have just x subvalues

    else:
        threads = threads_saved # recover the settings or the passed argument

    while dance <= 5: # iterate from 1 to 5

        request = 0

        for lol in MAX_LOL: # lol iterate

            for thread_n in range(threads): # MAX threads

                t = threading.Thread(target=do_something)
                t.setDaemon(True)
                t.start()

                request += 1

            main_thread = threading.currentThread()
            for t in threading.enumerate():
                if t is main_thread:
                    continue
                if request < len(MAX_LOL)-1 and settings.errors_count <= MAX_ERR_COUNT:
                    t.join()

        dance += 1

The code you see here it was cleaned because it was long to debug for you, so i try to semplify a little bit. As you can see there are many iteration , i start from a dbquery and i fetch the result in the list (iterations) next i adjust the max number of the threads allowed then i iterate again from 1 to 5 (it's an argoument passed to the small thread) then inside the value fetched from the query iteration there is a json that contain another list i need to iterate again ... and then finally i open the threads with start and join ...

The script open x threads and then, when they (all or almost) finish it will open others threads ... but my goal is to keep max X threads forever , i mean once one thread is finished another have to spawn and so on ... until the max_number of the threads is reached.

i hope you can help me. Thanks

ilmetu
  • 448
  • 11
  • 27
  • if this is the simplified version I think you need to try to clean up your actual code so that it is easier to read for yourself. – Tadhg McDonald-Jensen Apr 05 '16 at 14:03
  • i understand my code already :) i just don't want to poste all the code here ... but thanks for the suggestion. – ilmetu Apr 05 '16 at 15:01
  • 1
    I wasn't implying that you don't understand it I was implying that it is probably hard to read and possibly therefore harder to work with, and in any case I think you are looking for something like [a pool of threads](http://stackoverflow.com/a/3386632/5827215) working as much as they can. – Tadhg McDonald-Jensen Apr 05 '16 at 15:24
  • Thanks you point me in the right direction :) – ilmetu Apr 06 '16 at 13:24

0 Answers0