0

I have code where already loop is used and I want to replace loop with multithreading, but not able to understand how to do. below is code.

def run_test_function(arg1,arg2, arg3, arg4, arg5):
   try:
      #loop1
      for x in arg1:
         #loop2
         for y in arg2:
            #loop3
            for z in range(len(arg3)):
               #statements
            #statements
         #statements
   except Exception as e:
      print(e)
def main(argv):
   arg1 = value1
   arg2 = value2
   arg3 = value3
   arg4 = value4
   run_test_function(arg1, arg2, arg3, arg4)

I want every iteration of loop1 should behave as a thread, so that I can replace loop with multithreading. Appreciation and thanks in advance for help. :)

Adeep Malmotra
  • 51
  • 2
  • 10

1 Answers1

2

To print x and y in parallel (as you said in the comment):

from itertools import product
from multiprocessing import Pool


def run_test_function(arg1, arg2):
    with Pool(4) as pool:
        pool.starmap(print, product(arg1, arg2), 5)

if __name__ == '__main__':
    run_test_function([1,2,3,4,5], "abc")

Output:

~/test $ python so.py
1 a
1 b
1 c
2 a
2 b
2 c
3 a
3 b
3 c
4 a
4 b
4 c
5 a
5 b
5 c
6 a
6 b
6 c
7 a
7 b
7 c
ForceBru
  • 43,482
  • 10
  • 63
  • 98
  • I apologize but not able to understand how to implement it for my code. I have edited my question, can you please look into it and see what is actually happening with my code. – Adeep Malmotra Aug 10 '21 at 12:26
  • If `arg3` and `arg4` are constants, you can pass a lambda function: `pool.map(lambda x, y: processing_function(x, y, arg3, arg4), product(arg1, arg2))` – ForceBru Aug 10 '21 at 12:30
  • what is the functionality of product and Pool(4) and according to you I need to place the statements in different function and have to call that function in run_test_function? – Adeep Malmotra Aug 10 '21 at 12:33
  • "what is the functionality of `product` and `Pool(4)`" - you can find links to the documentation are right after the code in the answer. "I need to place the statements in different function and have to call that function in run_test_function?" - yes – ForceBru Aug 10 '21 at 12:50
  • thank you, you cleared a lot, one more question, what if there is one more nested loop as loop3 and loop3 is like for x in range(len(arg3)) ? I have edited my question accordingly. – Adeep Malmotra Aug 10 '21 at 13:02
  • Simply put the `range(len(arg3))` into `product`: `product(arg1, arg2, range(len(arg3)))`. `product` is specifically designed to let you replace `N` nested loops like `for a in A: for b in B: for c in C: ...` with a call to `product` with `N` arguments, like `for a, b, c, d, e in product(A, B, C, D, E): ...` – ForceBru Aug 10 '21 at 13:15
  • thank you, but I think product won't help me, bcoz I just want loop1's each iteration should behave as single thread. Can you please look into the question's edited part? I apologize for asking many questions, but I think I am progressing on my problem and getting clarified after every step I edit. – Adeep Malmotra Aug 10 '21 at 13:26
  • Using the same lambda function trick and assuming that all `arg`s except maybe `arg1` are constants: `pool.map(lambda x: processing_function(x, arg2, arg3, arg4), arg1)`, where `processing_function` loops over `arg2, arg3, arg4` sequentially – ForceBru Aug 10 '21 at 14:10
  • thank you, Can you convert the same code in python 2.7 – Adeep Malmotra Aug 17 '21 at 09:16
  • @AdeepMalmotra, ~~there are tools that can do this automatically~~ Oops, these tools convert Python 2.7 to Python 3, though. Anyway, this is a different question now – ForceBru Aug 17 '21 at 09:18
  • I have raised new question, can you please reply on the this link: https://stackoverflow.com/questions/68814939/converting-multiprocessing-python-3-code-to-python-2 – Adeep Malmotra Aug 17 '21 at 09:32