Share a variable among Threads which spawned from Processes in Python

Question

I would like share a variable with the main thread of a process spawned from the main of program and thread spawned from the same process.

Variable I want to share is not a simple object like array or int. This a socket zmq.socket object for my case but it could be any python object. This shared variable should only be accessible in the spawned process so declaring as a global variable is not acceptable.

Thanks in advance.

def process_function():
    
    
    # print init
    pid = os.getpid()
    print("PROCESS STARTS", pid)

    var = #this want to use in thread
    # create a thread
    thread = threading.Thread(target=thread_function)
    thread.start()
    ... # stuff
    thread.join()

def thread_function():

    # print start prompt
    print("THREAD STARTS")
    
    #do stuff with var

# In main
if __name__ == '__main__': 
        p = multiprocessing.Process(target=process_function, args=())
        p.start()
        #do stuff
        p.join()

EDIT 1

I have discovered a way using global keyword.

def process_function():
    global var # this is new
    #this var is only global within process, if you create
    #many processes each will have its own var

    # print init
    pid = os.getpid()
    print("PROCESS STARTS", pid)

    var = #this want to use in thread
    # create a thread
    thread = threading.Thread(target=thread_function)
    thread.start()
    ... # stuff
    thread.join()

def thread_function():
    global var #this is new as well
    # print start prompt
    print("THREAD STARTS")
    
    #do stuff with var

# In main
if __name__ == '__main__': 
        p = multiprocessing.Process(target=process_function, args=())
        p.start()
        #do stuff
        p.join()

score 2 · Answer 1 · answered Jan 05 '21 at 12:55

2

please see below eg. i have passed variable from main to process_function and process_function to thread_function import multiprocessing, os import threading

def process_function(input):
    print('2. inside process_function')
    print(input)
    pid = os.getpid()
    print("PROCESS STARTS", pid)
    var =  'variable from process func to thread'
    # create a thread
    thread = threading.Thread(target=thread_function, args=(var,))
    thread.start()
    thread.join()


def thread_function(msg):
    print('3. inside thread_function')
    # print start prompt
    print("THREAD STARTS")
    print(msg)

    # do stuff with var


# In main
if __name__ == '__main__':
    print('1. inside main')
    inputs = 'varaible from main'
    p = multiprocessing.Process(target=process_function, args=(inputs,))
    p.start()
    # do stuff
    p.join()

Output:

1. inside main
2. inside process_function
varaible from main
PROCESS STARTS 19372
3. inside thread_function
THREAD STARTS
variable from process func to thread

answered Jan 05 '21 at 12:55

mukund ghode

252
1
7

with increasing complexity, if you want to pass multiple args to multipocessing, you will have to use pool.map() or pool.startmap(). there are many other ways to do also. you can refer https://stackoverflow.com/questions/25553919/passing-multiple-parameters-to-pool-map-function-in-python/25553970 https://stackoverflow.com/questions/5442910/how-to-use-multiprocessing-pool-map-with-multiple-arguments – mukund ghode Jan 05 '21 at 13:03
for passing multiple args to multithreading,you can refer https://stackoverflow.com/questions/6904487/how-to-pass-a-variable-by-name-to-a-thread-in-python – mukund ghode Jan 05 '21 at 13:03
you can give it a try, if still fails, then you can tell the error to look more into your scenario! – mukund ghode Jan 05 '21 at 13:04
Thanks, your solution uses argument passing rather than using the variable defined the upper scope.In python when basic objects like integers, booleans passed as a parameter, both the caller and caller have their own copy of the variable. For example, you give var inside process as a parameter to the thread function. If var changes inside the thread, process cannot learn this change. In my scenario, I was exchanging a socket and a boolean. Bollean was controling workflow of the process, ie. shouldTerminate.If I pass it as an argument then changes on the boolean does not reflected – Gokberk Yar Jan 05 '21 at 18:49

Share a variable among Threads which spawned from Processes in Python

1 Answers1