how to control popen stdin, stdout, stderr redirection?

Question

I am confused about how popen() redirects stdin, stdout and stderr of the child process in unix. The man page on popen() is not very clear in this regard. The call

FILE *p = popen("/usr/bin/foo", "w");

forks a child process and executes a shell with arguments "-c", "/usr/bin/foo", and redirects stdin of this shell (which is redirected stdin of foo), stdout to p. But what happens with stderr? What is the general principle behind it?

I noticed that, if I open a file in foo (using fopen, socket, accept etc.), and the parent process has no stdout, it gets assigned the next available file number, which is 1 and so on. This delivers unexpected results from calls like fprintf(stderr, ...).

It can be avoided by writing

FILE *p = popen("/usr/bin/foo 2>/dev/null", "w");

in the parent program, but are their better ways?

Answer 1

popen(3) is just a library function, which relies on fork(2) and pipe(2) to do the real work.

However pipe(2) can only create unidirectional pipes. To send the child process input, and also capture the output, you need to open two pipes.

If you want to capture the stderr too, that's possible, but then you'll need three pipes, and a select loop to arbitrate reads between the stdout and stderr streams.

There's an example here for the two-pipe version.

Answer 2

simple idea: why not add "2>&1" to the command string to force the bash to redirect stderr to stdout (OK, writing to stdin still is not possible but at least we get stderr and stdout into our C program).

Answer 3

The return value from popen() is a normal standard I/O stream in all respects save that it must be closed with pclose() rather than fclose(3). Writing to such a stream writes to the standard input of the command; the command's standard output is the same as that of the process that called popen(), unless this is altered by the command itself. Conversely, reading from a "popened" stream reads the command's standard output, and the command's standard input is the same as that of the process that called popen().

From its manpage, so it allows you to read the commands standard output or write into its standard input. It doesn't say anything about stderr. Thus that is not redirected.

If you provide "w", you will send your stuff to the stdin of the shell that is executed. Thus, doing

FILE * file = popen("/bin/cat", "w");
fwrite("hello", 5, file);
pclose(file);

Will make the shell execute /bin/cat, and pass it the string "hello" as its standard input stream. If you want to redirect, for example stderr to the file "foo" do this first, before you execute the code above:

FILE * error_file = fopen("foo", "w+");
if(error_file) {
    dup2(fileno(error_file), 2);
    fclose(error_file);
}

It will open the file, and duplicate its file-descriptor to 2, closing the original file descriptor afterwards.

Now, if you have your stdout closed in your parent, then if the child calls open it will get 1, since that's (if stdin is already opened) the next free file-descriptor. Only solution i see is to just use dup2 and duplicate something into that in the parent, like the above code. Note that if the child opens stdout, it will not make stdout open in the parent too. It stays closed there.

Answer 4

Check out popenRWE by Bart Trojanowski. Clean way to do all 3 pipes.

Answer 5

if you just want to get STDERR, try this:

#include <stdio.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/wait.h>
#include <malloc.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>

/*
 * Pointer to array allocated at run-time.
 */
static pid_t    *childpid = NULL;

/*
 * From our open_max(), {Prog openmax}.
 */
static int      maxfd;

FILE *
mypopen(const char *cmdstring, const char *type)
{
    int     i;
    int     pfd[2];
    pid_t   pid;
    FILE    *fp;

    /* only allow "r" "e" or "w" */
    if ((type[0] != 'r' && type[0] != 'w' && type[0] != 'e') || type[1] != 0) {
        errno = EINVAL;     /* required by POSIX */
        return(NULL);
    }

    if (childpid == NULL) {     /* first time through */
        /* allocate zeroed out array for child pids */
        maxfd = 256;
        if ((childpid = calloc(maxfd, sizeof(pid_t))) == NULL)
            return(NULL);
    }

    if (pipe(pfd) < 0)
        return(NULL);   /* errno set by pipe() */

    if ((pid = fork()) < 0) {
        return(NULL);   /* errno set by fork() */
    } else if (pid == 0) {                          /* child */
        if (*type == 'e') {
            close(pfd[0]);
            if (pfd[1] != STDERR_FILENO) {
                dup2(pfd[1], STDERR_FILENO);
                close(pfd[1]);
            }
        } else if (*type == 'r') {
            close(pfd[0]);
            if (pfd[1] != STDOUT_FILENO) {
                dup2(pfd[1], STDOUT_FILENO);
                close(pfd[1]);
            }
        } else {
            close(pfd[1]);
            if (pfd[0] != STDIN_FILENO) {
                dup2(pfd[0], STDIN_FILENO);
                close(pfd[0]);
            }
        }

        /* close all descriptors in childpid[] */
        for (i = 0; i < maxfd; i++)
            if (childpid[i] > 0)
                close(i);

        execl("/bin/sh", "sh", "-c", cmdstring, (char *)0);
        _exit(127);
    }

    /* parent continues... */
    if (*type == 'e') {
        close(pfd[1]);
        if ((fp = fdopen(pfd[0], "r")) == NULL)
            return(NULL);
    } else if (*type == 'r') {
        close(pfd[1]);
        if ((fp = fdopen(pfd[0], type)) == NULL)
            return(NULL);

    } else {
        close(pfd[0]);
        if ((fp = fdopen(pfd[1], type)) == NULL)
            return(NULL);
    }

    childpid[fileno(fp)] = pid; /* remember child pid for this fd */
    return(fp);
}

int
mypclose(FILE *fp)
{
    int     fd, stat;
    pid_t   pid;

    if (childpid == NULL) {
        errno = EINVAL;
        return(-1);     /* popen() has never been called */
    }

    fd = fileno(fp);
    if ((pid = childpid[fd]) == 0) {
        errno = EINVAL;
        return(-1);     /* fp wasn't opened by popen() */
    }

    childpid[fd] = 0;
    if (fclose(fp) == EOF)
        return(-1);

    while (waitpid(pid, &stat, 0) < 0)
        if (errno != EINTR)
            return(-1); /* error other than EINTR from waitpid() */

    return(stat);   /* return child's termination status */
}

int shellcmd(char *cmd){
    FILE *fp;
    char buf[1024];
    fp = mypopen(cmd,"e");
    if (fp==NULL) return -1;

    while(fgets(buf,1024,fp)!=NULL)
    {
        printf("shellcmd:%s", buf);
    }

    pclose(fp);
    return 0;
}

int main()
{
    shellcmd("ls kangear");
}

and you will get this:

shellcmd:ls: cannot access kangear: No such file or directory