How to initialize a child process with passed in functions in Node.js

Question

Context

I'm building a general purpose game playing A.I. framework/library that uses the Monte Carlo Tree Search algorithm. The idea is quite simple, the framework provides the skeleton of the algorithm, the four main steps: Selection, Expansion, Simulation and Backpropagation. All the user needs to do is plug in four simple(ish) game related functions of his making:

1. a function that takes in a game state and returns all possible legal moves to be played
2. a function that takes in a game state and an action and returns a new game state after applying the action
3. a function that takes in a game state and determines if the game is over and returns a boolean and
4. a function that takes in a state and a player ID and returns a value based on wether the player has won, lost or the game is a draw. With that, the algorithm has all it needs to run and select a move to make.

What I'd like to do

I would love to make use of parallel programming to increase the strength of the algorithm and reduce the time it needs to run each game turn. The problem I'm running into is that, when using Child Processes in NodeJS, you can't pass functions to the child process and my framework is entirely built on using functions passed by the user.

Possible solution

I have looked at this answer but I am not sure this would be the correct implementation for my needs. I don't need to be continually passing functions through messages to the child process, I just need to initialize it with functions that are passed in by my framework's user, when it initializes the framework.

I thought about one way to do it, but it seems so inelegant, on top of probably not being the most secure, that I find myself searching for other solutions. I could, when the user initializes the framework and passes his four functions to it, get a script to write those functions to a new js file (let's call it my-funcs.js) that would look something like:

const func1 = {... function implementation...}
const func2 = {... function implementation...}
const func3 = {... function implementation...}
const func4 = {... function implementation...}

module.exports = {func1, func2, func3, func4}

Then, in the child process worker file, I guess I would have to find a way to lazy load require my-funcs.js. Or maybe I wouldn't, I guess it depends how and when Node.js loads the worker file into memory. This all seems very convoluted.

Can you describe other ways to get the result I want?

Answer 1

child_process is less about running a user's function and more about starting a new thread to exec a file or process.

Node is inherently a single-threaded system, so for I/O-bound things, the Node Event Loop is really good at switching between requests, getting each one a little farther. See https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/

What it looks like you're doing is trying to get JavaScript to run multiple threads simultaniously. Short answer: can't ... or rather it's really hard. See is it possible to achieve multithreading in nodejs?

So how would we do it anyway? You're on the right track: child_process.fork(). But it needs a hard-coded function to run. So how do we get user-generated code into place?

I envision a datastore where you can take userFn.ToString() and save it to a queue. Then fork the process, and let it pick up the next unhandled thing in the queue, marking that it did so. Then write to another queue the results, and this "GUI" thread then polls against that queue, returning the calculated results back to the user. At this point, you've got multi-threading ... and race conditions.

Another idea: create a REST service that accepts the userFn.ToString() content and execs it. Then in this module, you call out to the other "thread" (service), await the results, and return them.

Security: Yeah, we just flung this out the window. Whether you're executing the user's function directly, calling child_process#fork to do it, or shimming it through a service, you're trusting untrusted code. Sadly, there's really no way around this.

Answer 2

Assuming that security isn't an issue you could do something like this.

// Client side
<input class="func1"> // For example user inputs '(gamestate)=>{return 1}'
<input class="func2">
<input class="func3">
<input class="func4">

<script>
  socket.on('syntax_error',function(err){alert(err)});
  submit_funcs_strs(){
    // Get function strings from user input and then put into array
    socket.emit('functions',[document.getElementById('func1').value,document.getElementById('func2').value,...
  }
</script>

// Server side

// Socket listener is async
socket.on('functions',(funcs_strs)=>{
  let funcs = []
  for (let i = 0; i < funcs_str.length;i++){
    try {
        funcs.push(eval(funcs_strs)); 
    } catch (e) {
        if (e instanceof SyntaxError) {
            socket.emit('syntax_error',e.message);
            return;
        }
    } 

  }
  // Run algorithm here
}