Multiprocessing module in python2.7 causing some issue

Question

I am using multiprocessing module in python2.7 and found to have some issue with it, which is currently going out of my head, as in how to resolve it.

The sample code that I am using is as follows:-

from multiprocessing import Pool

def func_eachcol(col_index):
    global list_of_lists

    for row in xrange(list_of_lists):
        list_of_lists[row][col_index] = 'Update Value'

class C1:
    def func_callmultiprocess(self):

        global list_of_lists
        col_len = len(cols)
        pool = Pool(col_len)
        for col in xrange(col_len):
            pool.apply_async(func_eachcol, args=(col))
        pool.close()
        pool.join()

        print list_of_lists

Basically, what i am doing, is on each col I am opening up a new process, and iterating through each row of list_of_lists, and then updating on that row,col in that very process.

The issue I am seeing here is that after multiprocessing is over, i cannot see the updated list_of_lists, infact it is the old list_of_lists.

but when i see, inside the multiprocess_function (func_eachcol), then I could see the values getting updated in that very index(row, col).

Please provide some remedy to this problem.

Added one more code, to simple what I wanted to achieve

from multiprocessing import Pool

def func_eachcol(col_index):
    print "Hello",
    global list

    print list,
    list[col_index] = -list[col_index]
    print list

class C1:
    def func_callmultiprocess(self):

        global list 
        list = [1,2,3]

        col_len = len(list)

        pool = Pool(col_len)
        for col in xrange(col_len):
            pool.apply_async(func_eachcol, args=(col,))

        pool.close()
        pool.join()

        print list

c = C1()
c.func_callmultiprocess()


**Ouput**
Hello [1, 2, 3] [-1, 2, 3]
Hello [1, 2, 3] [1, -2, 3]
Hello [1, 2, 3] [1, 2, -3]
[1, 2, 3]

Basically what i wanted the output in the end is [-1, -2, -3]. How is it possible.?

In function func_callmultiprocess, try by adding the line `global list_of_lists` — avasal, Jun 01 '12 at 07:07
@avasal: he is not altering the list_of_lists in func_callmultiprocess; he is misunderstanding what state is shared between processes. — Martijn Pieters, Jun 01 '12 at 07:11
If you want to share that global variable maybe you want to use *threads* instead of *processes* — Diego Navarro, Jun 01 '12 at 07:16
No I can't use thread, as no multiprocessing happens in there. Why is that not possible, I have one global list into the memory, and I am accessing each index of that list through multiprocess, and updating them. After multiprocessing over, it should reflect the updated list..?? — geek, Jun 01 '12 at 07:18
[`multiprocessing.Manager` code example](http://stackoverflow.com/a/9754423/4279) — jfs, Jun 01 '12 at 07:25
your first code snippet contains several errors. It won't work even in a single process. — jfs, Jun 01 '12 at 07:28

score 1 · Accepted Answer · answered Jun 01 '12 at 07:08

1

Global variables are not shared between multiple processes; you are manipulating a global list in a sub-process only, the parent process won't see the changes.

Review the documentation on sharing state for options on how to pass information between processes. You probably want a Manager instance to share your list.

answered Jun 01 '12 at 07:08

Martijn Pieters

1,048,767
296
4,058
3,343

I have added one more example to demonstrate what actually I wanted to achieve. Please give sample code, in order to achieve the same.? – geek Jun 01 '12 at 07:19
Manager instance is not working out for me.. You can check at your end too.. I have tried it.. – geek Jun 01 '12 at 07:22
Better update your question and tell us *how* it is now working out for you. – Martijn Pieters Jun 01 '12 at 07:22

Multiprocessing module in python2.7 causing some issue

1 Answers1

Linked