97

I have a bash script which calls several long-running processes. I want to capture the output of those calls into variables for processing reasons. However, because these are long running processes, I would like the output of the rsync calls to be displayed in the console in real-time and not after the fact.

To this end, I have found a way of doing it but it relies on outputting the text to /dev/stderr. I feel that outputting to /dev/stderr is not a good way of doing things.

VAR1=$(for i in {1..5}; do sleep 1; echo $i; done | tee /dev/stderr)

VAR2=$(rsync -r -t --out-format='%n%L' --delete -s /path/source1/ /path/target1 | tee /dev/stderr)

VAR3=$(rsync -r -t --out-format='%n%L' --delete -s /path/source2/ /path/target2 | tee /dev/stderr)

In the example above, I am calling rsync a few times and I want to see the file names as they are processed, but in the end I still want the output in a variable because I will be parsing it later.

Is there a 'cleaner' way of accomplishing this?

If it makes a difference, I am using Ubuntu 12.04, bash 4.2.24.

jww
  • 97,681
  • 90
  • 411
  • 885
Mendhak
  • 8,194
  • 5
  • 47
  • 64

5 Answers5

94

Duplicate &1 in your shell (in my example to 5) and use &5 in the subshell (so that you will write to stdout (&1) of the parent shell):

exec 5>&1
FF=$(echo aaa|tee >(cat - >&5))
echo $FF

This will print "aaa" two times, once because of the echo in the subshell, and the second time it prints the value of the variable.

In your code:

exec 5>&1
VAR1=$(for i in {1..5}; do sleep 1; echo $i; done | tee >(cat - >&5))
# use the value of VAR1
Josh Correia
  • 3,807
  • 3
  • 33
  • 50
Op De Cirkel
  • 28,647
  • 6
  • 40
  • 53
  • This worked, thanks! So in this case, is the '5' must be a file descriptor, I will read up and learn about these. – Mendhak Sep 17 '12 at 05:15
  • 4
    Shouldn't the descriptor be closed in the parent shell? – akhan Mar 11 '14 at 05:27
  • 2
    @akhan: I assume that's `exec 5>&-` then? – hakre Sep 29 '14 at 13:03
  • 9
    How about `FF=$(echo aaa | tee /dev/tty)`? [Source](http://stackoverflow.com/a/7902174/4725840) – knight42 Dec 30 '15 at 05:40
  • I guess on my machine, `5` refers to `/dev/fd/63`? `tee: /dev/fd/63: No such file or directory` – user1011471 Sep 29 '16 at 17:39
  • 1
    This does not preserve output colors. – Shantanu Bhadoria May 01 '17 at 00:57
  • 2
    @user1011471, when you write `tee >(inner_cmd...)`, the shell opens a pipe to the inner command, such that the pipe's write side is also inherited by the outer command (tee), typically under the descriptor 63. The outer command has no idea the shell did anything funny, it gets a file name argument `/dev/fd/63`. https://en.wikipedia.org/wiki/Process_substitution Don't know why you got this error, but it's unrelated to inner command writing to descriptor 5. – Beni Cherniavsky-Paskin Dec 07 '17 at 11:26
46

Op De Cirkel's answer has the right idea. It can be simplified even more (avoiding use of cat):

exec 5>&1
FF=$(echo aaa|tee /dev/fd/5)
echo $FF
Russell Davis
  • 8,319
  • 4
  • 40
  • 41
  • 9
    Wouldn't /dev/fd/5 be OS specific? – akhan Mar 11 '14 at 05:25
  • 12
    I had been wondering if you could just use `tee /dev/fd/1`, but that doesn't work because the output still gets captured by `$()`. So in case anyone else is wondering the same thing, it _is_ necessary to use an extra file descriptor (like 5). – dirtside Sep 20 '15 at 18:42
  • 2
    We could go simplifying even further and making this a oneliner without the `exec`: `{ FF=$(echo aaa|tee /dev/fd/5); } 5>&1 ` The braces allow for the redirection to happen before the subshell command is run, while `$FF` still remains in the scope of the current shell (that wouldn't work with normal brackets `(` `)`. This way there's even no need to close FD 5 afterwards, which is a overlooked hygienic habit. – tlwhitec May 11 '18 at 08:30
  • 2
    @akhan No it wouldn't, bash is said to be emulating this path should it not exist in the OS by itself. – tlwhitec May 11 '18 at 08:32
  • 1
    If this is run using `sudo -u – Paul Donohue Dec 13 '19 at 15:41
  • Why does @dirtside solution not work? Specifically, why can't you just do `tee /dev/fd/1`? I don't understand if you're going to `tee /dev/fd/5` where 5 points 1, how is that different than just `tee /dev/fd/1`? – Nick Aug 05 '23 at 05:50
21

Here's an example capturing stderr, stdout, and the command's exit code. This is building on the answer by Russell Davis.

exec 5>&1
FF=$(ls /taco/ 2>&1 |tee /dev/fd/5; exit ${PIPESTATUS[0]})
exit_code=$?
echo "$FF"
echo "Exit Code: $exit_code"

If the folder /taco/ exists, this will capture its contents. If the folder doesn't exist, it will capture an error message and the exit code will be 2.

If you omit 2>&1then only stdout will be captured.

Bryan Roach
  • 743
  • 9
  • 12
9

If by "the console" you mean your current TTY, try

variable=$(command with options | tee /dev/tty)

This is slightly dubious practice because people who try to use this sometimes are surprised when the output lands somewhere unexpected when they don't have a TTY (cron jobs etc).

tripleee
  • 175,061
  • 34
  • 275
  • 318
7

You can use more than three file descriptors. Try here:

http://tldp.org/LDP/abs/html/io-redirection.html

"Each open file gets assigned a file descriptor. [2] The file descriptors for stdin, stdout, and stderr are 0, 1, and 2, respectively. For opening additional files, there remain descriptors 3 to 9. It is sometimes useful to assign one of these additional file descriptors to stdin, stdout, or stderr as a temporary duplicate link."

The point is whether it's worth to make script more complicated just to achieve this result. Actually it's not really wrong, the way you do it.

Piotr Wadas
  • 1,838
  • 1
  • 10
  • 13