1

When I am running my python code that uses multiprocessing, it is using all 8 cores and my system is hanging. I added the following line, but it didn't help. 

po = multiprocessing.Pool(processes=4)

recursive code is as follows:

def func(a,i):
   if (a>i):
      func(a-1,i)
      func(a-5,i)
   else print a

Now above 2 recursive calls can be put into threads as they are independent of each other.

Below is the code.

import multiprocessing
po = multiprocessing.Pool(processes=2)

p=bin(47)
q=bin(59)


def func(r,i):
   if( p[i]==r[i])  
    po = multiprocessing.Process(target=func, args=(2*r, i+1))
    po.start()
    po = multiprocessing.Process(target=func, args=((2*r)+1, i+1))
    po.start()

func(1,1)
Pratibha
  • 1,730
  • 7
  • 27
  • 46
  • 1
    Please provide more information. How exactly didn't it help? How many processes are created? If more than 4 (or 5, including the main one), you'll need to show more code. – Lev Levitsky Dec 13 '12 at 12:12
  • Well, thinking about it I assume that your processor have 4 cores but its up to 8 threads (4 cores X 2 threads each core), so making up 4 processes will consume the 8 cores. In my opinion it could be an answer. – Netwave Dec 13 '12 at 12:16
  • My system is i7 that means it have 8 CPUs. number of processes to be created is more than 2^150. i am working on number theory and working on n bit number. I implemented code using threading in python but threads are blocking each other when thread count is more than 2^20. So I used multiprocessing. – Pratibha Dec 13 '12 at 12:21
  • 1
    2^150 threads is quite challenging. – kaspersky Dec 13 '12 at 12:27
  • yes but its according to bit size. when using threading, if active threads are more than 10,000 I make current thread to sleep for 60 sec. but I dont understand why still they are blocking each other. how can I synch the threads to operate in queue. – Pratibha Dec 13 '12 at 12:38
  • What do you mean by "blocking each other", do you mean the threads are in deadlock? – kaspersky Dec 13 '12 at 12:42
  • Even if I am using processes=1, it is using all 8 CPUs. – Pratibha Dec 13 '12 at 12:42
  • threads shouldnt be in deadlock as they are not using any global variable but when I put to print threading.activeCount() and make current thread sleep if active threads > 10000, after sometime it become contant to 11247 or similar value and it continue to that for hours. – Pratibha Dec 13 '12 at 12:57
  • 1
    I think you might find the question [Python thread pool similar to the multiprocessing Pool?](http://stackoverflow.com/q/3033952/355230) helpful. – martineau Dec 13 '12 at 13:55
  • Please show some code and explain why you think you need a huge number of threads. There is surely a better approach. – Janne Karila Dec 18 '12 at 14:18
  • I have edited the question giving basic code without threads – Pratibha Dec 19 '12 at 06:24
  • Your recursive function returns either instantly or never - if a is smaller than i, you just print i and then it instantly returns, if not you increase a and make a recursive call, where of course a is still larger than i. Even the second recursive call will never get executed because the first one never returns... – l4mpi Dec 19 '12 at 07:13
  • Sorry I made the changes required, please see the edited code. – Pratibha Dec 19 '12 at 10:19
  • You may have implemented a fork bomb. See [here](http://stackoverflow.com/questions/2697640/multiprocessing-bomb) – Simon Bergot Dec 19 '12 at 10:20
  • @Simon, but the threads will stop once we get required values – Pratibha Dec 19 '12 at 10:25
  • @pratibha. Can you post your code? If you are doing `po = multiprocessing.Pool(processes=4)` at the top level, then no matters what you do, the multiprocessing lib will recursively spawn 4 processes without even calling your function. We really need your code to help you. – Simon Bergot Dec 19 '12 at 10:32
  • @Simon, code is available now. – Pratibha Dec 19 '12 at 10:44

0 Answers0