Trying to copy back elements in Array works with integers but not with strings in Multiprocessing Python module

Question

I am trying to write a process which does some computation on an Array filled with strings using the multiprocessing module. However, I am not able to get back the results. This is just a minimalist code example:

from multiprocessing import Process, Value, Array
from ctypes import c_char_p

# Process
def f(n, a):
    for i in range(0,10):

        a[i] = "test2".encode('latin-1')

if __name__ == '__main__':

    # Set up array 
    arr = Array(c_char_p, range(10))

    # Fill it with values
    for i in range(0,10):
        arr[i] = "test".encode('latin-1')



    x = []

    for i in range(0,10):
        num = Value('d', float(i)*F)
        p = Process(target=f, args=(num, arr,))
        x.append(p)
        p.start()


    for p in x:
        p.join()

    # THis works
    print(num.value)

    # This will not give out anything
    print(arr[0])

The last line won't print out anything, despite it being filled or altered.

The main thing that concerns me, is when changing the code to just simply using integers it will work:

from multiprocessing import Process, Value, Array
from ctypes import c_char_p

def f(n, a):

    for i in range(0,10):

        a[i] = 5
if __name__ == '__main__':



    arr = Array('i',range(10))
    for i in tqdm(range(0,10)):
        arr[i] = 10



    x = []

    for i in range(0,10):
        num = Value('d', float(i)*F)
        p = Process(target=f, args=(num, arr,))
        x.append(p)
        p.start()


    for p in x:
        p.join()

    print(num.value)
    print(arr[0])

My Best guess is that this has something to do with the fact that the string array is acutally filled with char arrays and an integer is just one value, but I do not know how to fix this

score 2 · Answer 1 · answered Jul 02 '18 at 08:17

This might answer your question, Basically the string array arr has an array of character pointers c_char_p, When the first process invokes the function f the character pointers are created in the context of itself but not in the other processes context, so eventually when the other processes tries to access the arr it will be invalid addresses.

In my case this seems to be working fine,

from multiprocessing import Process, Value, Array
from ctypes import c_char_p
values = ['test2438']*10

# Process
def f(n, a):
    for i,s in enumerate(values):
            a[i] = s

if __name__ == '__main__':

    # Set up array 
    arr = Array(c_char_p, 10)
    for i in range(0,10):
    arr[i] = 'test'
    # Fill it with values
    x = []

    for i in range(0,10):
        num = Value('d', float(i))
        p = Process(target=f, args=(num, arr,))
        x.append(p)
        p.start()

    for p in x:
        p.join()

    # This will not give out anything
    print(arr[:])

Trying to copy back elements in Array works with integers but not with strings in Multiprocessing Python module

1 Answers1