11

I have a program that reads from two input files simultaneously. I'd like to have this program read from standard input. I thought I'd use something like this:

$program1 <(cat) <($program2)

but I've just discovered that

cat <(cat)

produces

....
mmap2(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb758e000
read(0, 0xb758f000, 131072)             = -1 EIO (Input/output error)
....
cat: -: Input/output error

and similarly,

$ cat <(read -n 1)
bash: read: read error: 0: Input/output error

So... Linux is failing to read at the syscall level. That's interesting. Is bash not wiring stdin to the subshell? :(

Is there a solution to this? I specifically need to use process substitution (the ... <(...) format) because $program1 (tail, incidentally) expects files, and I need to do some preprocessing (with od) on standard input before I can pass it to tail - I can't just specify /dev/stdin et al.

EDIT:

What I actually want to do is read from a file (which another process will be writing to) while I also read from standard input so I can accept commands and such. I was hoping I could do

tail -f <(od -An -vtd1 -w1) <(cat fifo)

to read from standard input and the FIFO simultaneously and drop that into a single stdout stream I could run through awk (or similar). I know I could solve this trivially in any scripting language, but I like learning how to make bash do everything :P

EDIT 2: I've asked a new question that more fully explains the context I described just above.

Community
  • 1
  • 1
i336_
  • 1,813
  • 1
  • 20
  • 41
  • 3
    Interesting because it works on OS-X. So not coming from bash as I initially suspected. That's an interesting one. – Daniel Voina Mar 17 '17 at 05:39
  • @DanielVoina: Thanks for the info, I was very curious about other platforms! – i336_ Mar 17 '17 at 06:09
  • I don't quite understand your real need as it's a bit weird to `tail` the `stdin`. Hope my answer can give you some hint. – pynexj Mar 17 '17 at 10:33
  • 2
    Seeing questions like this on StackOverflow gives me hope for the programming profession – Leo Izen Mar 17 '17 at 10:50
  • @whjm: Very good point. I [fully described the context in a new question](http://stackoverflow.com/questions/42859785/how-can-i-select-ie-simultaneously-read-from-standard-input-and-a-file-in), and accepted your answer to this one as it does canonically answer it :) – i336_ Mar 17 '17 at 14:13
  • @DanielVoina -- The behavior is not OS related. Maybe OSX is using a different (older?) version of bash which deals with pgrps in a different (buggy?) way. – pynexj Mar 17 '17 at 14:30
  • @whjm: GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16) is the standard OS-X version – Daniel Voina Mar 17 '17 at 17:06
  • @DanielVoina -- could you help try `ssh user@host` on your osx as i mentioned in my another answer? – pynexj Mar 18 '17 at 15:31
  • @whjm: On ssh session the behaviour is also flawed see here: http://pastebin.com/BRQCDV2G – Daniel Voina Mar 18 '17 at 16:07

2 Answers2

13

1. Explain why cat <(cat) produces EIO

( I'm using Debian Linux 8.7, Bash 4.4.12 )

Let's replace <(cat) with the long running <(sleep) to see what's happening.

From pty #1:

$ echo $$
906
$ tty
/dev/pts/14
$ cat <(sleep 12345)

Go to another pty #2:

$ ps t pts/14 j
  PPID    PID   PGID    SID TTY       TPGID STAT   UID   TIME COMMAND
   903    906    906    906 pts/14    29999 Ss       0   0:00 bash
   906  29998    906    906 pts/14    29999 S        0   0:00 bash
 29998  30000    906    906 pts/14    29999 S        0   0:00 sleep 12345
   906  29999  29999    906 pts/14    29999 S+       0   0:00 cat /dev/fd/63
$ ps p 903 j
  PPID    PID   PGID    SID TTY       TPGID STAT   UID   TIME COMMAND
     1    903    903    903 ?            -1 Ss       0   0:07 SCREEN -T linux -U
$

Let me explain it (according to the APUE book, 2nd edition):

  1. The TPGID being 29999 indicates that cat (PID 29999) is the foreground process group which is now controlling the terminal (pts/14). And sleep is in the background process group (PGID 906).
  2. The process group of 906 is now an orphaned process group because "the parent of every member is either itself a member of the group or is not a member of the group’s session". (The PID 906's PPID is 903 and 903 is in a different session.)
  3. When the process in an orphaned background process group reads from its controlling terminal, read() would fail with EIO.

2. Explain why cat <(cat) sometimes works (not really!)

Daniel Voina mentioned in a comment that cat <(cat) works on OS X with Bash 3.2.57. I just managed to reproduce it also on Linux with Bash 4.4.12.

From pty #1:

bash-4.4# echo $$
10732
bash-4.4# tty
/dev/pts/0
bash-4.4# cat <(cat)
cat: -: Input/output error
bash-4.4#
bash-4.4#
bash-4.4# bash --norc --noprofile  # start a new bash
bash-4.4# tac <(cat)
                      <-- It's waiting here so looks like it's working.

(The first cat <(cat) failing with EIO was explained in the first part of my answer.)

Go to another pty #2:

bash-4.4# ps t pts/0 j
 PPID   PID  PGID   SID TTY      TPGID STAT   UID   TIME COMMAND
10527 10732 10732 10732 pts/0    10805 Ss       0   0:00 bash
10732 10803 10803 10732 pts/0    10805 S        0   0:00 bash --norc --noprofile
10803 10804 10803 10732 pts/0    10805 S        0   0:00 bash --norc --noprofile
10804 10806 10803 10732 pts/0    10805 T        0   0:00 cat
10803 10805 10805 10732 pts/0    10805 S+       0   0:00 tac /dev/fd/63
bash-4.4# ps p 10527 j
 PPID   PID  PGID   SID TTY      TPGID STAT   UID   TIME COMMAND
10526 10527 10527 10527 ?           -1 Ss       0   0:00 SCREEN -T dtterm -U
bash-4.4#

Let's see what's happening:

  1. The TPGID being 10805 indicates that tac (PID 10805) is the foreground process group which is now controlling the terminal (pts/0). And cat (PID 10806) is in the background process group (PGID 10803).

  2. But this time the pgrp 10803 is not orphanded because its member PID 10803 (bash)'s parent (PID 10732, bash) is in another pgrp (PGID 10732) and it's in the same session (SID 10732).

  3. According to the APUE book, SIGTTIN will be "generated by the terminal driver when a process in a (non-orphaned) background process group tries to read from its controlling terminal". So when cat reads stdin, SIGTTIN will be sent to it and by default this signal would stop the process. That's why the cat's STAT column was shown as T (stopped) in the ps output. Since it's stopped the data we input from keyboard are not sent to it at all. So it just looks like it's working but it's not really.

Conclusion:

So the different behaviors (EIO vs. SIGTTIN) depend on whether the current Bash is a session leader or not. (In the 1st part of my answer, the bash of PID 906 is the session leader, but the bash of PID 10803 in the 2nd part is not the session leader.)

pynexj
  • 19,215
  • 5
  • 38
  • 56
  • Why does this happen? What is 903 and in what session do we find it? – Emil Vikström Mar 17 '17 at 07:58
  • Added info about `903`. But here all we need to know is it's in a different session. – pynexj Mar 17 '17 at 08:00
  • **a)** Headsmash; now I have to figure out a second approach. **b)** Thankyou so much for this amazing information :) (PS: GNU sleep accepts undocumented `infinity` parameter.) – i336_ Mar 17 '17 at 10:36
  • I never know that. Not as easy as inputting a big number. :) – pynexj Mar 17 '17 at 10:39
  • Silly question but, why can't we use /dev/stdin and combine it with a pipe? Something like `some_filtering_cmd | real_cmd /dev/stdin <(other_cmd) | tail` etc. – Leo Izen Mar 17 '17 at 10:48
  • @whjm: Very true! `sleep 12345` and `sleep 9d` are less keystrokes :) – i336_ Mar 17 '17 at 13:08
  • @LeoIzen: I asked [a new question](http://stackoverflow.com/questions/42859785/how-can-i-select-ie-simultaneously-read-from-standard-input-and-a-file-in) with full context and details, FYI – i336_ Mar 17 '17 at 14:32
1

The accepted answer explained why, but I saw one solution which can solve it. It is by subshelling it with additional (), such as:

(cat <(cat))

Please find the solution details here: https://unix.stackexchange.com/a/244333/89706

Johnny Wong
  • 945
  • 9
  • 16