1

How can I tell when the process started by popen is done?

I'm passing the descriptor (from fileno() called on the FILE * returned by popen) to a function which calls fstat() and uses the value to returned to determine how much to read. Surprisingly this actually works if there is a delay (for instance, stepping through the debugger) but returns a file size of zero if called immediately after the popen. So I need some way to wait until all the output is ready - how can I do this? I assume I can't call pclose() even though it does this waiting, because then the descriptor isn't valid any more.

Update: Actually, it appears the code works fine if I pretend the fstat() failed on the pipe call - the issue seems to be that fstat() does not fail, as expected. So what I really want is a way to tell if the descriptor is a pipe - fstat returns st_mode=0 (was expecting S_IFIFO!)

Michael
  • 9,060
  • 14
  • 61
  • 123

5 Answers5

3

If you don't read from the pipe, the subprocess is fairly likely to never terminate. The buffer between the ends of the pipe is fairly small. So, as the subprocess writes output, the buffer will fill and the subprocess will block in the write() call. Then you'll have deadlock. The parent process is waiting for the subprocess to terminate before reading from the pipe, the subprocess is blocked until the parent process reads from the pipe and so can't terminate.

Ken Thomases
  • 88,520
  • 7
  • 116
  • 154
2

This won't work (at least not on Linux system).

The file descriptor (as given by fileno(3)) of a popen(3)-ed command is a pipe(7). So it is not seekable, and fstat(2) won't give any significant size on it.

If you need such fine grained information, don't use popen but call the underlying syscalls pipe(2), fork(2), execve(2), waitpid(2) then you can use poll(2).

If you insist on using popen you could still poll its fileno; however there is no standard way to get its pid (to use waitpid) this is why you should use the syscalls(2) directly.

I guess that fstat-ing a pipe should not give a plain file but a fifo(7) if it succeeds, so special-case on st.st_mode & S_IFMT == S_IFIFO to detect that case.

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • That was my expectation too. Apparently OS X (BSD) deviates in this regard, as fstat() will return the exact size of output, if I wait until the process is done. – Michael Apr 10 '15 at 05:47
  • 1
    @Michael: That's not going to be reliable because pipes do not have unlimited size. It's going to cap out at the max pipe buffer size at which point the program will block waiting for you to read. – R.. GitHub STOP HELPING ICE Apr 10 '15 at 05:51
  • @R.. Is there a way to tell whether or not fstat() is returning meaningful info? I actually would prefer it to fail if the input is a pipe, but it isn't. – Michael Apr 10 '15 at 05:52
  • 2
    @Michael: Just disregard the `st_size` (or treat the whole operation as failure) when `S_ISREG(st_mode)` is false (i.e. when the fd is not a regular file). – R.. GitHub STOP HELPING ICE Apr 10 '15 at 05:55
1

In unix-like OSes a parent process recieves SIGCHLD when its children terminate. See this tutorial

Also you can select() on the pipe descriptor to check if there's a data to read. When the pipe is closed a "read-ready" event with zero-size data is generated.

Community
  • 1
  • 1
user3159253
  • 16,836
  • 3
  • 30
  • 56
1

Perhaps you have some special reason for wanting to determine the amount of data available on the pipe from popen, but this is not generally available and not the intended usage case. popen (in read mode) is intended to be used with programs that output a stream of data as fast as they can (note: they'll block on the pipe when its buffer fills up if you don't read it instantly) and then exit once they're done, producing EOF for your reader. This gives you pretty much flexibility; you can use line-based read functions like fgets or getline, read the while stream until EOF, or read large chunks with fread and be prepared for a short read on the last one.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • I guess I was just expecting fstat() to fail and handle that case differently - the problem seems to be that it *doesn't* fail. If I pretend it did, then everything works fine when the input descriptor is a pipe. – Michael Apr 10 '15 at 05:51
  • If you want to use `popen`, use it correctly and detect completion via `EOF` on the `FILE` stream. If you want to wait for process termination without reading the pipe then you need to make the child process yourself and don't use `popen`, but then you're going to run into trouble when the pipe buffer fills up and the process never exits on its own without you first reading some data out of the pipe. – R.. GitHub STOP HELPING ICE Apr 10 '15 at 05:53
0

I don't know if this will help but it's a similar situation where I want to store the output of a popen().

void live(void) { // scan and read the dongle
  FILE *dong;  // dongle
  char *c;
  char cmd[] = "rtl_power -f 88M:108M:5k -1 -";
  c = buf;  // externally allocated buffer
  dong = popen(cmd,"r");
  while (!feof(dong)) {
    *c = fgetc(dong);
    c++;
  }
  pclose(dong);
  textsize = (int) (c-buf);
  printf("Read %i bytes\n",textsize);
}

It sort of tails the file, copying it to buf[]. It's been working for a few days, I've even published the program on Sourceforge. Running Linux, Debian and Raspbian.

Alan Corey
  • 577
  • 6
  • 10
  • [**Why is “while (!feof(file))” always wrong?**](https://stackoverflow.com/questions/5431941/why-is-while-feoffile-always-wrong) `feof()` will not return non-zero until **after** the EOF condition is reached, and that won't happen until `fgetc()` fails. And note that `fgetc()` returns `int`, not `char`. In the code you have posted, the last `fgetc()` that fails when EOF is reached returns an `int` typed-`EOF` value that is truncated to a `char`, and incorrectly stuffed into `buf`. Only after that happens does `feof()` return true. – Andrew Henle May 07 '19 at 22:34
  • The feof() condition isn't true until after the read that causes it. The same thing happens with fgets, which is safe as long as you check a second time for feof() before you use the data. Loop until feof but realize you may have just read an EOF so don't use it if feof() is true. EOF is 0xFFFF I think, cast to a char it's just 0xFF. In the code above feof() is true when the piped process ends. – Alan Corey May 08 '19 at 23:41
  • Actually above feof() is true once the piped process stops and you've read all the data it generated. Supposedly it's buffered somewhere but with some small buffer and I didn't want to lose any by assuming I could read it later. It's a chicken or egg situation almost. There's much about C that isn't pretty. – Alan Corey May 09 '19 at 00:21
  • EOF is probably really -1, so it can appear as 0xff, 0xffff, 0xffffffff depending where it is. Many file-related functions return -1 on error,and maybe that's what EOF is. Does it exist in the file on the disk? Never looked but I wouldn't depend on it. I think it's a condition that exists when you've run out of data. – Alan Corey May 09 '19 at 01:14