3

Let's say I have an executable called myProgram that reads from an input file and writes to an output file. A command line for this program looks like this:

./myProgram -o outputfile inputfile

The argument to the -o option specifies an output file name; if no such file exists then the program will create it.

What I was wondering is whether I can execute myProgram multiple times, piping the output of the one instance to the input of the next. For example, 

./myProgram inputfile | ./myProgram -o outputfile

Is it possible to achieve this and if so, what would I have to implement? exec calls and forking? just simple read and write calls?

John Bollinger
  • 160,171
  • 8
  • 81
  • 157
Kevag6
  • 135
  • 1
  • 1
  • 10
  • This is standard command line behavior. Check the source of any coreutils program e.g. [`cat`](http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/cat.c#n655). – Max Nov 17 '17 at 16:04

4 Answers4

3

Pipes work by chaining stdin and stdout of multiple programs together.

You need to modify your program to have the ability to read from stdin and write to stdout instead of to specific files. Then you can use pipes.

dbush
  • 205,898
  • 23
  • 218
  • 273
  • So when the second myprogram is executed it should read from the stdout of the first myprogram run correct? Or is my logic wrong? – Kevag6 Nov 17 '17 at 16:02
  • @Kevag6 Correct. The first instance in the pipe writes to stdout, and the second instance in the pipe reads from stdin. – dbush Nov 17 '17 at 16:03
  • ... and the program does not need to know or care, @Kevag6, where its `stdin` came from. In particular, it doesn't have to do anything special to handle the case where its stdin is connected to a pipe. – John Bollinger Nov 17 '17 at 16:04
  • @JohnBollinger thank you for clarifying, I have one more question, in the 2nd case above which ./myprogram would get executed first? the first or the second? Thanks again – Kevag6 Nov 17 '17 at 18:03
  • 1
    Typically, pipes are opened from right to left. So the second would execute and be waiting on input, then the first would execute and start generating output which the second would read. So typically both programs would be running at the same time. – dbush Nov 17 '17 at 18:20
2

The short answer is: yes, you can do this.

However, your program should then read its input from stdin and write to stdout. This means, arguments as input or output file will no longer be necessary.

./myProgram < inputfile > outputfile

or, to illustrate the chaining,

./myProgram < inputfile | ./myProgram > outputfile

A widely spread combination is to read from stdin if no input file is specified and write to stdout if no outputfile or outputfile "-" is passed at the command line. This provides maximum flexibility.

Having said that, if it is sensible to chain multiple instances of the same program depends of course largely on what this program does. For example, for a sorting program this doesn't seem to make much sense of course ;)

Ctx
  • 18,090
  • 24
  • 36
  • 51
1

There is absolutely no problem to run two instances of your program myProgram together as you do with command 

./myProgram inputfile | ./myProgram -o outputfile
Pierre François
  • 5,850
  • 1
  • 17
  • 38
  • Well, it's no problem *for the shell*, but whether that has the OP's desired effect depends on the program. I think the details of that dependency are what he's asking about. – John Bollinger Nov 17 '17 at 16:07
0

You do not need to change your program.

In Unix, everything is (represented as) a file, including stdout and stdin.

If your program outputs to stdout, then running

./myProgram inputfile | ./myProgram -o outputfile /dev/stdin

will work. This uses stdin as the input "file" to the second program.

If your program requires some output file and does not normally output to stdout (not sure where else it'd output to, but whatever), then you can use

./myProgram -o /dev/stdout inputfile | ./myProgram -o outputfile /dev/stdin

to force it to output to the stdout "file", and then pipe that to the next command.

If you do want to change your program...

Make sure that it reads from stdin and outputs to stdout when no arguments are provided. Since you're programming in C, fgets() and fwrite() should suffice in most cases (and notice both of these commands are again reading from/writing to the file descriptors for stdin and stdout).

éclairevoyant
  • 392
  • 2
  • 15