0

I'd like to use multiprocessing.Process on a function that takes an ndarray type object in its argument.

def func(array):
     do_something_to array
return array

Is this possible?

EDIT: this is the code I tried to implement. The function 'noflux' takes a few ndarray type objects and makes changes to the arguments and this is the function I'd like to run on multiple processors.

for k in np.arange(0,len(t)-1,1):
    for j, j in enumerate(gridd):        
    
            i = int(j[1])
            j = int(j[0])

            # NO FLUX BOUNDARIES
            if cells[j,i] == 1:

                p1 = mp.Process(target=noflux,args=(j,i,cells,rfu,lfu,ufu,dfu,))
                p2 = mp.Process(target=noflux,args=(j,i,cells,rfh,lfh,ufh,dfh,))
                
                p1.start()
                p2.start()
                
                p1.join()
                p2.join()

EDIT 2: Here are the type of objects I pass to the function...

j,i are integers, cells is an ndarray shape (73,87), rfu,lfu,ufu,dfu are also (73,87) ndarrays

the function noflux takes these arrays and modifies them but it does not return anything. There's also a function within noflux that is defined and used so maybe this is what's causing the problem?

EDIT 3: Full error message received when running my code. Swapped the real path to file for "PATH"

Traceback (most recent call last):

  File "C:\PATH", line 418, in <module>
    p1.start()

  File "C:\anaconda\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)

  File "C:\anaconda\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)

  File "C:\anaconda\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)

  File "C:\anaconda\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)

  File "C:\anaconda\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)

BrokenPipeError: [Errno 32] Broken pipe
Conor McCann
  • 11
  • 2
  • Did you try it? – Marc Jul 06 '20 at 11:46
  • I did, and received a "BrokenPipeError: [Errno 32] Broken Pipe" error message. I will edit my question to include the code I tried to use. – Conor McCann Jul 07 '20 at 14:03
  • Why do you comment out the last line? and could you show what type of objects you pass into the function? – Marc Jul 10 '20 at 12:27
  • Passing ndarrays to a process should not be a problem so something else is probably giving you problems here – Marc Jul 10 '20 at 12:34
  • Hi, thanks for the response and that last line wasn't supposed to be commented out but I'm still getting the same error as before. I made an edit in the post to show what kinds of objects I'm passing :) – Conor McCann Jul 12 '20 at 14:45
  • could you change the lines `for j, j in enumerate(gridd): i = int(j[1]) j = int(j[0])` to `for l, subgrid in enumerate(gridd): i = int(grid[1]) j = int(grid[0])` and see if that solves anything? – Marc Jul 13 '20 at 09:36
  • Trying it I received ```name 'grid' is not defined ``` and if you meant 'gridd' instead, I should mention that ```gridd.shape = (3142,2)``` and is an array of coordinates used in the loop and so something like ```gridd[1,x]``` would be needed to specify a number from this array. I also tried ```j = int(gridd[l,0])``` and ```i = int(gridd[l,1])``` but again received a broken pipe error. – Conor McCann Jul 13 '20 at 15:35
  • Ah sorry, I meant `for l, subgrid in enumerate(gridd): i = int(subgrid[1]) j = int(subgrid[0])` It's kind of hard to determine what the problem is as your code here does not suggest a problem aside from your use of j for three different variables. Could you post a more elaborate error message? – Marc Jul 13 '20 at 19:34
  • Haha, yeah the j thing is a bit stupid I know... I tried again and got the same error message which I will post in an edit. – Conor McCann Jul 14 '20 at 13:08
  • Right now I can think of two things happening that you can try. First, since you are on windows could you verify that you implement the issue described in this answer https://stackoverflow.com/a/53924048/5516057? If that does not work could you try and run it in combination with the solution given here https://stackoverflow.com/a/33599967/5516057 and see if an error is raised from your process? – Marc Jul 15 '20 at 08:33

0 Answers0