1

I am trying to pipe a large array from child process to parent process. It is about 200,000 * 100 * 2 number of elements in an array. I possibly need to pass even greater value in the future.

The below is my child process for passing the array to the parent process

process.on('message', (msg) => {
    // console.log('inside child class');
    excelParserTool(msg).then((result)=>{
        process.stdout.write(JSON.stringify(result));
        process.exit();
    })
});

result is the large array that it sends to the parent process

the below is the parent process which receives the array and processes it.

excelParserChild.stdout.on('data', (data) => {
    excelBuffer += data
});

excelParserChild.stderr.on('data', (error) => {
    console.log('stderr');
    console.log(error);
    errorFinal += error;
    console.log(errorFinal)
});
excelParserChild.on('close', (exitCode) => {
   console.log(excelBuffer);

})

This causes 

(node:1468) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): RangeError: Invalid string length
(node:1468) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code

I have tried using process.send(result) to try to send the array also but it causes the same error. Now I am wondering if this even is the correct way of passing large data from child to parent process. What is the best methodology of passing large variable from child process to parent process? If I am doing it right, how do I solve invalid string length issue? Thanks.

forJ
  • 4,309
  • 6
  • 34
  • 60
  • Possible duplicate of [Pass large array to node child process](https://stackoverflow.com/questions/44052913/pass-large-array-to-node-child-process) – Javad Aug 21 '17 at 05:28
  • Invalid string length, means you are creating a string that is too large for the system to handle. Do you really need to process all of that at once, or could you break it up, retrieve / use smaller parts at different times? – Patrick Evans Aug 21 '17 at 05:28

0 Answers0