1

My goal is to save the session of some process execution to json (smth like [{ type: 'stdout', data: 'What's your name?' }, { type: 'stdin', data: 'Alex.' }, { type: 'stdout', data: 'Hi, Alex!' }]). I decided to do it with nodejs, but I've run into some problems. The stdout of spawned process works differently when being piped and when being inherited. So, for example I have this C code (simple guess number game) compiled to main:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main(void) {
  int random_num = 0;
  int guessed_num = 0;
  int counter = 0;

  srand(time(NULL));
  random_num = rand() % 10 + 1;

  printf("Guess my number! ");

  while(1) {
    counter++;

    scanf("%d", &guessed_num);

    if (guessed_num == random_num) {
      printf("You guessed correctly in %d tries! Congratulations!\n", counter);
      break;
    }

    if (guessed_num < random_num)
      printf("Your guess is too low. Guess again. ");

    if (guessed_num > random_num)
      printf("Your guess is too high. Guess again. ");
  }

  return 0;
}

And this JavaScript code:

var spawn = require('child_process').spawn;
var child = spawn('./main', { stdio: 'inherited' });

The result of execution of this JavaScript code will be:

Guess my number! 1
Your guess is too low. Guess again. 2
Your guess is too low. Guess again. 3
You guessed correctly in 3 tries! Congratulations!

However, when stdio is inherited, I can't attach to any of process streams and save the data to json. So I tried this:

var spawn = require('child_process').spawn;
var child = spawn('./main', { stdio: 'pipe' });
child.stdout.on('data', function(data) { process.stdout.write(data) });
process.stdin.on('data', function(data) { child.stdin.write(data) });

And get this as a result of execution:

1
2
3
Guess my number! Your guess is too low. Guess again. Your guess is too low. Guess again. You guessed correctly in 3 tries! Congratulations!

The stdout of child process is somehow hangs when the child process is waiting for input. Probably it has to do something with event-loop, but I'm not sure. Anyway, the behavior of inherited and pipe is very different and it seems wrong...

What can I do about it? Maybe there is some workaround?

alexb
  • 880
  • 11
  • 16

1 Answers1

1

In inherited mode, the C program inherits of a descriptor bound to the terminal. The terminal being an interactive device, it automatically sets the buffering of the printf() functions to stdout to be line buffered (i.e. the buffer is flushed when a new line is encountered).

OTOH, in pipe mode, the libc doesn't detect an interactive device, and switches to full buffering (i.e. the buffer is only flushed when its full or when fflush() is called.

A workaround would be to call fflush(stdout) after each printf() call or to disable buffering for stdout alltogether: setvbuf(stdout, NULL, _IONBF, 0);.

See this answer for a full explanation of pipes vs. terminal buffering, including an example of libc determining the buffering for stdout.

Community
  • 1
  • 1
Frederik Deweerdt
  • 4,943
  • 2
  • 29
  • 31
  • Thank you, it explains a lot. I didn't pay attention to this thing in Node documentation: "Note, however, that some programs use line-buffered I/O internally. While that does not affect Node.js, it can mean that data sent to the child process may not be immediately consumed." Another option I see is to use a PTY (e.g. `pty.js`), but in this case the ability to distinguish stdout and stderr will be lost. So, it seems impossible to emulate an interactive device and distinguish stdout and stderr streams. – alexb May 15 '16 at 11:37
  • However, running this same C program within a Docker container with (dockerode)[https://github.com/apocas/dockerode] it's actually possible to demultiplex streams. I've read the source of it and it seems that it connects to `/var/run/docker.sock` socket and then demultiplex the stream that Docker exports. So it seems that Docker handles it somehow... I just don't understand how :) – alexb May 15 '16 at 11:46
  • Wow, I found an easier solution. `spawn('stdbuf', ['-i0', '-o0', '-e0', './main'], { stdio: 'pipe' })` works just fine! Changing C code is fine, but I don't always have sources of it, so I think this solution is more appropriate. Anyway, thanks a lot! – alexb May 15 '16 at 12:18