4

I would like to run a program "A", have its output go to the input to another program "B", as well as stdin going to intput of "B". If program "A" closes, I'd like "B" to continue running.

I can redirect A output to B input easily:
./a | ./b

And I can combine stderr into the output if I'd like:
./a 2>&1 | ./b

But I can't figure out how to combine stdin into the output. My guess would be:
./a 0>&1 | ./b
but it doesn't work.

Here's a test that doesn't require us to rewrite up any test programs:

$ echo ls 0>&1 | /bin/sh -i
$ a  b  info.txt
$
/bin/sh: Cannot set tty process group (No such process)

If possible, I'd like to do this using only bash redirection on the command line (I don't want to write a C program to fork off child processes and do anything complicated everytime I want to do some redirection of stdin to a pipe).

markie
  • 41
  • 1
  • 2
  • maybe the `tee` command will help you. Good luck. – shellter Jun 23 '11 at 14:45
  • As I think about it, @markie, I wonder what you're trying to do. Is this a case where sourcing a script (e.g. `. ~/script.sh`) will do what you want? – Arthur Shipkowski Jun 24 '11 at 02:00
  • @Arthur This came up when playing a linux programming game. http://www.overthewire.org/wargames/vortex/ I haven't made it very far yet, but two levels so far run into this issue. Basically, if you give the program a correct value, it uses exec() or the like to give you a shell "/bin/sh -i". For some reason bash acts really weird with its input redirected like this. If it runs out of input before I give it more, it can act strange, which have led to clunky hacks http://axtaxt.wordpress.com/2010/12/28/overthewire-vortex-level10-2/ I have a feeling we're missing an obvious solution. – markie Jun 24 '11 at 14:02

6 Answers6

3

This cannot be done without writing an auxiliary program.

In general, stdin could be a read-only file descriptor (heck, it might refer to read-only file). So you cannot "insert" anything into it.

You will need to write a "helper" program that monitors two file descriptors (say, 0 and 3) in order to read from both and "merge" them. A simple select or poll loop would be sufficient, and you could write it in most scripting languages, but not the shell, I don't think.

Then you can use shell redirection to feed your program's output to descriptor 3 of the "helper".

Since what you want is basically the opposite of "tee", I might call it "eet"...

[edit]

If only you could launch "cat" in the background...

But that will fail because background processes with a controlling terminal cannot read from stdin. So if you could just detach "cat" from its controlling terminal and run it in the background...

On Linux, "setsid cat" should do it, roughly. But (a) I could not get it to work very well and (b) I really do not have time for this today and (c) it is non-standard anyway.

I would just write the helper program.

[edit 2]

OK, this seems to work:

{ seq 5 ; sleep 2 ; seq 5 ; } | /bin/bash -c 'set -m ; setsid cat ; echo HELLO'

The set -m thing forces bash to enable job control, which apparently is needed to prevent the shell from redirecting stdin from /dev/null.

Here, the echo HELLO represents your "program A". The seq commands (with the sleep in the middle) are just to provide some input. And yes, you can pipe this whole thing to process B.

About as ugly and non-portable a solution as you could ask for...

Nemo
  • 70,042
  • 10
  • 116
  • 153
  • I don't think even a background `cat` will read from multiple descriptors *simultaneously*. (If it doesn't need to be simultaneous, a foreground usage will do the trick, per my answer.) I'd hoped `socat` had something useful, but it did not. – Arthur Shipkowski Jun 23 '11 at 16:22
  • @Arthur: Right, but with "cat" running in the background copying stdin to stdout for you, the script could then proceed to write its own data to stdout or run a subprocess that does so. – Nemo Jun 23 '11 at 17:33
  • I see.. use cat to take the input of stdin and echo it as an output. Once converted to an output, it should theoretically be able to be piped somewhere. Neat idea. Let's ignore the simultaneity issue here, can it be done then? The ideas below building on this don't quite work for some reason. – markie Jun 23 '11 at 18:37
  • @Arthur @markie: I have added a second update with a construct that appears to do the trick. But it's hideous. – Nemo Jun 23 '11 at 19:05
  • @Nemo I figured that in the background you couldn't count on the `stdout`s being the same, but you seem to have made it work...in an ugly way. I think I'd rather have the C program! – Arthur Shipkowski Jun 24 '11 at 01:56
  • It's OK to have two processes share a stdout. The trick is making sure it gets redirected _before_ either process runs... Which it will as long as the whole mess is wrapped in that `bash -c`. But yeah, an auxiliary program would be cleaner. Perl or Python would do as well as C for this purpose. – Nemo Jun 24 '11 at 02:09
  • @Nemo I gave up and went ahead to write a program that forks to start program B as a child with the stdin and stdout changed to pipes I can control as I please. Then the program continues as program A, and always forwards its stdin to the child stdin pipe. Works great for 'normal' programs. But if program B is "/bin/sh -i", it acts really wonky. It will exit on its own instead of responding to further commands. Even worse is when program B is just a wrapper which starts up '/bin/sh -i' using exec or system or the like. It can break out of my pipes and start printing to the screen on its own. – markie Jun 24 '11 at 14:12
1

A pipe has two ends. One is for writing, and that which gets written appears in the other end, which is for reading.

It's a pipe, not a T or Y junction.

I don't think your scenario is possible. Having "stdin going to input of" anything doesn't make sense.

unwind
  • 391,730
  • 64
  • 469
  • 606
  • I was able to attach both stdout and stderr to the pipe no problem, so it is possible to have more than one thing attached to it. Just like you can redirect multiple things to the same file as well. And I don't understand why you complain about "stdin going to input of". Maybe my english is bad. When I run "cat", stdin is going to the input of cat. Does that explain what I mean by "stdin going to input of" better? Now I'd like stdin to come out of a pipe as well to connect to the input of a program. – markie Jun 23 '11 at 18:34
1

If I understand your requirements correctly, you want this set up (ASCII art to the fore):

o----+----->|  A   |----+---->|  B  |---->o
     |                  ^
     |                  |
     +------------------+

with the additional constraint that if process A closes up shop, process B should be able to continue with the input stream going to B.

This is a non-standard setup, as you realize, and can only be achieved by using an auxilliary program to drive the input to A and B. You end up with some interesting synchronization issues but it will all work remarkably well as long as your messages are short enough.

The plumbing necessary to achieve this is notable - you'll need two pipes, one for the input to A and the other for the input to B, and the output of A will be connected to the input of B as well.

o---->|  C  |---------->|  A   |----+---->|  B  |---->o
         |                          ^
         |                          |
         +--------------------------+

Note that C will be writing the data twice, once to A and once to B. Note, too, that the pipe from A to B is the same pipe as the pipe from C to A.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
1

To make the given test case work you have to while ... read from the controlling terminal device /dev/tty inside a sh -c '...' construct.

Note the use of eval (could it be avoided here?) and that multi-line commands on input> will fail.

echo 'ls; export var=myval'  | ( 
stdin="$(</dev/stdin)"
/bin/sh -i -c '
  eval "$1"; 
  while IFS="" read -e -r -p "input> " line; do 
    history -s "${line}"
    eval "${line}"; 
  done </dev/tty
' argv0 "${stdin}"
)

input> echo $var

For a similar problem and the use of named pipes see here:

BASH: Best architecture for reading from two input streams

Community
  • 1
  • 1
tim
  • 11
  • 1
0

This can't be done exactly as shown, but to perform your example you can make use of cat's ability to join files together:

cat <(echo ls) - | /bin/sh

(You can do -i, but then you'll have to have another process kill the /bin/sh, as your attempts to Ctrl-C and Ctrl-D out will fail.)

This assumes that you want to pass in your piped input and then accept from stdin. You can also make it so that it does something after stdin is done, or on both sides -- but it won't merge input character-by-character or line-by-line.

Arthur Shipkowski
  • 3,606
  • 1
  • 22
  • 30
  • That is a neat idea, but acts very strange. I can't see the shell prompt or previous commands when I press 'up'. But I can type and run commands. To exit I need to type 'exit' and press enter twice. What causes these weird things? – markie Jun 23 '11 at 18:26
  • Oh, and with -i I can see the shell prompt, but it doesn't really respond to what I type (Ctrl C makes it print the shell prompt again, enter or any commands are just ignored). What!? – markie Jun 23 '11 at 18:27
  • @markie: Without the -i, the shell isn't interactive, and so it runs in a mode more suitable for a script. Scripts don't want prompts, and don't need previous commands via 'up'. I'm not sure about the exit aspect, actually, as I didn't observe that. With -i it's trying to connect to a terminal, but the problem is that a pipeline doesn't act like a terminal -- Ctrl-C generates no signal, for example, instead being passed along as yet another character for processing. – Arthur Shipkowski Jun 24 '11 at 01:54
  • @markie: More specifics on interactive bash: http://www.gnu.org/software/bash/manual/bashref.html#Interactive-Shell-Behavior -- note that a lot of the solutions, including mine, are likely bash-specific! – Arthur Shipkowski Jun 24 '11 at 02:01
0

This seems to do what you want:

$ ( ./a <&-; cat ) | ./b

(It's not clear to me if you want a to get input...this solution sends all input to b) Of course, in this case the inputs to b are strictly ordered: all of the output of a is sent to b first, then a terminates, then input goes to b. If you want things interleaved, try:

$ ( ./a <&- & cat ) | ./b
William Pursell
  • 204,365
  • 48
  • 270
  • 300
  • This is a neat idea, but I tried your solution with the test case given in the problem and I couldn't get it to work. "(echo ls; cat ) | /bin/sh -i" does appear to give me a shell, and I can type, but it doesn't accept anything I type. What is going on here? – markie Jun 23 '11 at 18:20