4

I want to redirect the output of stdout and stderr to a common file:

./foo.sh >stdout_and_stderr.txt 2>&1

But also redirect just stderr to a separate file. I tried variations of:

./foo.sh >stdout_and_stderr.txt 2>stderr.txt 2>&1

but none of them work quite right in bash, e.g. stderr only gets redirected to one of the output files. It's important that the combined file preserves the line ordering of the first code snippet, so no dumping to separate files and later combining.

Is there a neat solution to this in bash?

CyGuy
  • 71
  • 5
  • 2
    I don't think you are going to be able to do that with a single process, and certainly not by simply twisting redirections as shown in the question. You may be able to get to it with piping something to `tee`, but the interleaving that results will likely be different from what you'd see on the terminal screen because of buffering, etc. – Jonathan Leffler Sep 11 '15 at 22:06
  • Note that your first command (`./foo.sh 2>&1 >file`) sends errors to the original standard output, and the standard output (but not the redirected standard error) to the file. If you wanted both in `file`, you'd have to use `./foo.sh >file 2>&1`, reversing the order of the redirections. They're interpreted reading left to right. – Jonathan Leffler Sep 11 '15 at 22:14
  • Right good point, just fixed the 2>&1 ordering, thanks @JonathanLeffler – CyGuy Sep 11 '15 at 22:17

3 Answers3

3

You can use an additional file descriptor and tee:

{ foo.sh 2>&1 1>&3- | tee stderr.txt; } > stdout_and_stderr.txt 3>&1

Be aware that line buffering may cause the stdout output to appear out of order. If this is a problem, there are ways to overcome that including the use of unbuffer.

Dennis Williamson
  • 346,391
  • 90
  • 374
  • 439
  • This looks like a simple one-liner, but can't get it to work. I get "bash: syntax error near unexpected token `}'". Removing the semi-colon took away the error, but when I try to cat the files, I get 'bash: 3: Bad file descriptor' – CyGuy Sep 14 '15 at 17:00
  • 1
    @CyGuy: If you don't include the opening curly brace or omit the space after it you'll get that error. – Dennis Williamson Sep 14 '15 at 17:47
  • Indeed works when respecting spaces @DennisWilliamson – CyGuy Sep 14 '15 at 21:19
  • @DennisWilliamson, I don't see how POSIX semantics allow unbuffer or any other tool to determine ordering of writes between stderr and stdout when those are not copies of the *exact same file descriptor*. If you've got a FIFO involved, the amount of delay involved between a write to the FIFO and the program on the other end being scheduled to read from it (and then, in the case of `tee`, perform another write with the content that it read) is indeterminate, no matter how immediate the flush at the front of the chain is. – Charles Duffy Jun 12 '18 at 20:45
1

Using process substitution, you can get a moderate approximation to what you're after:

file1=stdout.stderr
file2=stderr.only
: > $file1    # Zap the file before starting
./foo.sh >> $file1 2> >(tee $file2 >> $file1)

This names the files since one of the names is repeated. The standard output is written to $file1. Standard error is written to the pipeline, which runs tee and writes one copy of the input (which was standard error output) to $file2, and also writes a second copy to $file1 as well. The >> redirections mean that the file is opened with O_APPEND so that each write will be done at the end, regardless of what the other process has also written.

As noted in comments, the output will, in general, be interleaved differently in this than it would if you simply ran ./foo.sh at the terminal. There are multiple sets of buffering going on to ensure that is what happens. You might also get partial lines because of the ways lines break over buffer size boundaries.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
0

This comment from @jonathan-leffler should be an answer:

Note that your first command (./foo.sh 2>&1 >file) sends errors to the original standard output, and the standard output (but not the redirected standard error) to the file.

If you wanted both in file, you'd have to use ./foo.sh >file 2>&1, reversing the order of the redirections.

They're interpreted reading left to right.

BaZZiliO
  • 232
  • 1
  • 6
  • 1
    My comment about the redirection in the question was valid, but doesn't solve the problem. The problem is how to get one write function call to write to two different files simultaneously, and the short answer is "you can't". So, the standard error has to be written somewhere that can make two copies. – Jonathan Leffler Sep 11 '15 at 22:25