1

Can I use multiple threads to execute a single for loop in python? I mean, my loop has huge number of iterations and so I would like to run multiple threads simultaneously in such a way that each thread can execute a certain number of iteration of the loop.

Assume that the number of iterations is in high orders of magnitude. 10**9 for example.

And also, if this is possible, does it reduce the time it requires to complete the execution of the for loop....?

Salai V V
  • 15
  • 1
  • 7
  • Suggest you watch the talk https://www.youtube.com/watch?v=MCs5OvhV9S4 The global interpreter lock can be a problem for CPU bound tasks when multiple threads are used. David goes through various techniques you can use. – Graham Dumpleton Jun 21 '15 at 13:02

1 Answers1

0

I don't have a lot of experience using Python, but more in general: Usage of multiple threads is only easy to implement if the iterations in the loop are independent of each other. Which means that the calculation inside an iteration should not depend on the result of the previous one.

If the iterations are independent you could split up the work by giving each worker thread 'a part' of the job. This can be achieved by giving each thread the same loop, but with different start and stop conditions. So using multiple threads will result in completing the loop faster (how much depends on your processor).

This topic might be of help: How to use threading in Python?

Community
  • 1
  • 1
Leon Noordam
  • 73
  • 6
  • Oh. Alright man. And in my case, the results depend on the previous iteration. But, I think I can still implement your idea by coming up with a little analysis. Anyways, thanks a lot........ – Salai V V Jun 21 '15 at 14:55