0

I have Monte-Carlo tree search implementation for some games. I would like to accelerate rollout phase with using multiprocessing. But I get the folLowing error:

RuntimeError: RLock objects should only be shared between processes through inheritance.

To get a minimally reproducible example of the error, I changed the game class to a tree search with random numbers instead of real positions.

MCTS.py code:

from numpy import random 
from copy import deepcopy
import multiprocessing as mp

class Node:
    def __init__(self, Pos) -> None:
        self.Pos = Pos
        self.visit = 0
        self.child = []
        self.flagTerminate = False
        self.flagInit = False
        if random.random() < 0.5 and Pos != 0 :
            self.flagTerminate = True
        
class MCTS:
    def __init__(self, dictLockerNodes) -> None:
        self.currNode = Node( 0 )
        self.dictPos = {}
        self.dictPos[ self.currNode.Pos ] = self.currNode
        dictLockerNodes[ self.currNode ] = mp.RLock()

    def __initChildren(self, node: Node, dictLockerNodes, LockerDictPos ):
        with dictLockerNodes[ node ]:
            if node.flagInit :
                return
            validMoves =  range(3)
            for move in validMoves:
                tempPos =  deepcopy( node.Pos )
                tempPos = tempPos + move  #without making move
                
                with  LockerDictPos :
                    tempNode =  self.dictPos.get( tempPos, None)
                    if tempNode is None:
                        tempNode = Node( tempPos )
                        self.dictPos[ tempPos ] = tempNode
                        dictLockerNodes[ tempNode ] = mp.RLock()
                
                    node.child[move] = tempNode
            
            node.flagInit = True
    
    def __updateNode (self, node: Node, res, Locker):
        with Locker:
            node.visit += res 
    
    def __simulation (self, node: Node, dictLockerNodes, LockerDictPos):
        if node.flagTerminate :
            return 1  #without real result
            
        self.__initChildren( node, dictLockerNodes , LockerDictPos)
        validMoves =  range(3)
        selectedMove = random.choice(validMoves)
        res = self.__simulation( node.child[selectedMove], dictLockerNodes, LockerDictPos  )
        self.__updateNode( node, res, dictLockerNodes[node])
        return 1  #without real result
    
    def runSimulation (self, move, dictLockerNodes, LockerDictPos):
        self.__initChildren( self.currNode, dictLockerNodes, LockerDictPos )
        res =  self.__simulation( self.currNode[move], dictLockerNodes, LockerDictPos )
        self.__updateNode( self.currNode, res, dictLockerNodes[self.currNode] )

launch.py code:

import multiprocessing as mp
from MCTS import MCTS, Node

dictLockerNodes = {}
LockerDictPos = mp.RLock()

Bot = MCTS( dictLockerNodes )

if __name__ == '__main__':
    Moves = range(3)
    
    ArgMoves = [ (move, dictLockerNodes, LockerDictPos ) for move in Moves  ]

    with mp.Pool( processes =  3 ) as pool:
        pool.map( Bot.runSimulation, ArgMoves )
        pool.close()
        pool.join()

I tried to changes initialisation mp.Pool according to this question:

Lock objects should only be shared between processes through inheritance

Maybe I didn't catch something, but it doesn't work for my code.

Update: The following idea also doesn't work:


import multiprocessing as mp
from MCTS import MCTS, Node

if __name__ == '__main__':

    manager = mp.Manager()
    dictLockerNodes = manager.dict()
    LockerDictPos = manager.RLock()
    procs = []

    Bot = MCTS( dictLockerNodes)
    Moves = range(3)
    for move in Moves:
        proc = mp.Process( target= Bot.runSimulation, args=( move, dictLockerNodes, LockerDictPos) )
        procs.append(proc)
        proc.start()

    [ proc.join() for proc in procs ]
    [ proc.close() for proc in procs  ]
ets_ets
  • 1
  • 1
  • @larsks In which part of the code should I use Manager? – ets_ets Jul 18 '23 at 17:10
  • `manager.RLocks` are useless because they work on threads, and manager objects themselves use threading to answer requests to the managed objects. You would need to patch `manager.RLock` to use your last approach correctly as described here: https://stackoverflow.com/a/73206773/16310741 – Charchit Agarwal Jul 18 '23 at 20:49
  • 1
    I'm not sure if your code is multiprocessing compatible. In multiprocessing, all variables (except shared memory) are per-process instances. Updating a variable in one of the processes will not be reflected in any of the other processes. This means that the `Bot` instance will never be updated. If this is what you expect, you can simply create one `Bot` instance per process, then RLock is no longer needed. – ken Jul 19 '23 at 03:45
  • @ken , maybe I need to rewrite this code. My goal is to build and update Monte Carlo tree in parallel mode. How I can do it? I would like to use GPU with pytorch multiprocessing – ets_ets Jul 19 '23 at 11:17

0 Answers0