0

Followup to

Given that the obvious use of coproc does not work as I expected, as seen in:

$ cat test.sh
coproc cat auto/etc/build.cfg
while read -u ${COPROC[0]} BRANCH TARGET SVNSRC SVNTAG BUILDTYPE DISTTYPE DISTARGS
do
    echo hello
done

$ bash -x test.sh
+ read -u 63 BRANCH TARGET SVNSRC SVNTAG BUILDTYPE DISTTYPE DISTARGS
+ cat auto/etc/build.cfg
+ echo hello
hello
+ read -u BRANCH TARGET SVNSRC SVNTAG BUILDTYPE DISTTYPE DISTARGS
test.sh: line 2: read: BRANCH: invalid file descriptor specification

Question: Why does the coproc go away after the script reads one line of output?

Community
  • 1
  • 1
Jim Garrison
  • 85,615
  • 20
  • 155
  • 190

1 Answers1

3

I cannot reproduce:

bash-4.1 $ cat infile 
one
two
three
four
five

bash-4.1 $ cat s.sh 
coproc cat infile
while read -u ${COPROC[0]} v; do
  echo "$v"
done

bash-4.1 $ bash -x s.sh 
+ read -u 63 v
+ cat infile
+ echo one
one
+ read -u 63 v
+ echo two
two
+ read -u 63 v
+ echo three
three
+ read -u 63 v
+ echo four
four
+ read -u 63 v
+ echo five
five
+ read -u 63 v
+ echo ''

+ read -u 63 v

Edit: I did reproduce it like this:

bash-4.1 $ cat s.sh 
coproc cat infile

sleep 1

while read -u ${COPROC[0]} v; do
  echo "$v"
done

bash-4.1 $ bash  -x s.sh 
+ sleep 1
+ cat infile
+ read -u v
s.sh: line 5: read: v: invalid file descriptor specification

Edit: See comments below.

It seems that the co-process times out quickly ... May be your system is slow :)

No, the command executed as co-process is too quick, if you slow it down, it works:

bash-4.1 $ cat s.sh 
coproc while read -r; do
  printf '%s\n' "$REPLY"
  sleep 1
done < infile

sleep 1

while read -u ${COPROC[0]} v; do
  echo "$v"
done

bash-4.1 $ bash s.sh 
one
two
three
four
five

Anyway, I believe that this test case is not appropriate. You need a co-process when you need a two-way pipe (i.e. you need to chat with the co-process). You can use a single database connection (the database connections are resource expensive) and go back and forth with your queries and shell code.

Edit (see comments below). The issues related to the stdin buffering could be worked around with some non standard tools (in this case stdbuf is used (part of recent versions of the GNU coreutils, I believe):

~/t$ cat s
coproc stdbuf -oL -i0 mysql

printf '%s;\n' 'show databases' >&${COPROC[1]}

printf '\n\nshowing databases, fisrt time ...\n\n\n'

while read -t3 -u${COPROC[0]}; do
  printf '%s\n' "$REPLY"
  [[ $REPLY == test ]] && {
    printf '%s\n' 'test found, dropping it ...'
    printf '%s;\n' 'drop database test' >&${COPROC[1]}
    }
done

printf '\n\nshowing databases, second time ...\n\n\n'


printf '%s;\n' 'show databases' >&${COPROC[1]}

while read -t3 -u${COPROC[0]}; do
  printf '%s\n' "$REPLY"
done


printf '%s\n' quit >&${COPROC[1]}

Output:

~/t$ bash s


showing databases, fisrt time ...


Database
information_schema
mysql
sakila
test
test found, dropping it ...
world


showing databases, second time ...


Database
information_schema
mysql
sakila
world

I realize this approach has many drawbacks ...

Dimitre Radoulov
  • 27,252
  • 4
  • 40
  • 48
  • What happens if you do something more complex inside the loop, like `svn status`? I found that the above worked on Cygwin (bash 4.1.10) but not Fedora 14 (bash 4.1.7). However it failed on both with something more complex inside the loop. – Jim Garrison Oct 07 '11 at 15:29
  • "maybe your system is slow" -- it's a dual Xeon 2.6GHz quad-core (i.e. 8 full cores, not HT) with plenty of memory. ` load average: 0.00, 0.01, 0.05` – Jim Garrison Oct 07 '11 at 15:32
  • "the command executed as co-process is too quick" - this is not how I would have expected things to work, based on working with coprocesses in Java. Regardless of when the coprocess finishes, its output is buffered and available to be read until you reap the process (waitFor). I guess the paradigm is just different in bash, which after considering for a while I am beginning to understand. – Jim Garrison Oct 07 '11 at 15:40
  • 1
    Here's my new understanding of `coproc`: it is intended for true co-processes more on the model of network peers that run independently and exchange data. When one of the processes shuts down, all buffered but unread data is discarded. The problem occured because of my attempt to use this for a purpose that is not in the set of intended use cases. – Jim Garrison Oct 07 '11 at 15:45
  • I think this is valuable information for others seeking help with `coproc`. With your permission, I'd like to add the above "new understanding" text to your answer (and then accept it, of course). – Jim Garrison Oct 07 '11 at 15:46
  • Yes, I'm playing with it right now, the implementations of co-processes in bash is different, than, ksh's implementations, for example. And yes, the stdout buffering makes most shell access difficult. @JimGarrison, you can add the text, of course. – Dimitre Radoulov Oct 07 '11 at 16:08