Redirect sterr to file but leave all output on screen intact (as_is). [windows XP - BAT files]

Question

Pls, imagine that I have some program which has output in both main streams: err && out. As an example we can use next bat file:

@echo off
echo 1
echo 2 >&2
echo 3
echo 4 >&2
echo 5
echo 6 >&2
echo 7
echo 8 >&2
echo 9
echo 10 >&2

so on the console' screen I will see this output:

1
2
3
4
5
6
7
8
9
10

And I want to see this output! Exactly as I expected. But also I want to see whole err stream simultaneously at some specific 'ERR.out' file. So It's content taking into account my initial bat file should be:

2
4
6
8
10

So My question is - HOW to do that trick for windows XP SP3 in BAT file's scenario?

I would like to use next pseudo code - but it will not work of course. But I hope that the main logic will be the same: command.exe | print_it_as_is_on_screen | redirect_2_stream_to_ERR.out_file | send_whole_output_to_another_command.exe

Answer 1

I believe it is impossible to reliably do what you want. Below is my original answer that does not work, followed by a modification that may work, but is not guaranteed to work in all situations, followed by an explanation as to why it cannot be done perfectly.

There are 2 parts to solving your problem.

First, you need a tee program that can read stdin and write it to both stdout and a file. You can use tee.exe from gnuwin32 CoreUtils for Windows, or you could use the following hybrid JScript/batch file - TEE.BAT

@if (@X)==(@Y) @end /* Harmless hybrid line that begins a JScript comment

::--- Batch section within JScript comment that calls the internal JScript ----
@echo off
cscript //E:JScript //nologo "%~f0" %*
exit /b

----- End of JScript comment, beginning of normal JScript  ------------------*/
var fso = new ActiveXObject("Scripting.FileSystemObject");
var mode=2;
if (WScript.Arguments.Count()==2) {mode=8;}
var out = fso.OpenTextFile(WScript.Arguments(0),mode,true);
var chr;
while( !WScript.StdIn.AtEndOfStream ) {
  chr=WScript.StdIn.Read(1);
  WScript.StdOut.Write(chr);
  out.Write(chr);
}

Make sure tee.exe or tee.bat is either in your current folder, or else somewhere within your path.

You could then pipe the output from your batch script to tee as follows:

yourBatch | tee output.txt

But the pipe operation captures stdout and redirects it to the 2nd program. So the above line would capture stdout in output.txt, but you want to capture stderr instead.

Second, you need a mechanism to swap stdout and stderr. It is surprisingly easy :-)

yourBatch 3>&2 2>&1 1>&3 | tee err.txt

I describe how the above works in my accepted answer to Is there a way to redirect ONLY stderr to stdout (not combine the two) so it can be piped to other programs?

The above does not work :-(

There are multiple timing issues that prevent the above from working properly. The first issue is that there is a startup time before tee is ready to process the input. The batch file (or whatever process) may write interleaved messages to stdout and stderr, with stderr piped to tee, and stdout going directly to the console. The stdout is immediately forwarded to the console, but the stderr is delayed while tee prepares the input and output streams. In your test case, the entire stdout content is written to the console before tee even begins to write to the console. So the console output looks like

1
3
5
7
9
2
4
6
8
10

I am able to get the correct output on both Win 7 and also an XP virtual machine by using an additional batch script to delay launching the program until the tee has finished initializing. This only works with the gnu tee.exe. It does not work reliably with tee.bat.

First I create delay.bat in the same folder as the test folder.

@echo off
ping 192.0.2.2 -n 1 -w 1000 >nul
%*

Then the following command gives the correct results on my machine

delay.bat test.bat 3>&2 2>&1 1>&3 | tee.exe err.txt

The results are not reproducible if I substitute tee.bat for tee.exe. It may not work on other machines even with tee.exe.

Occasionally tee.bat produces the correct output, but usually one of the numbers is dropped, or else some stdout and stderr output gets merged into one line. The reason it is not reliable has to do with a more subtle timing issue.

The original program writes both stdout and stderr to the console just fine because it is one process controlling (writing) everyting. The process can only write one thing at a time. But when stderr is piped to TEE, then there are 2 processes trying to write to the console simultaneously. TEST.BAT is writing stdout, and TEE is writing stderr. There is nothing to stop the simultaneous writes from getting intermingled, resulting in mixed up output. I suppose to get truly simultaneous writes the machine must have multiple CPUs, or at least multiple cores. But even a single CPU machine can have problems because there is no way to keep the two processes synchronized. Even if the output does not get intermingled, it could easily get out of order.

Even though tee.exe works on my machine, I suspect it is possible to even get that to fail, either with different source input, or else on another machine.

I believe it is impossible to reliably get your results. You can redirect stderr to stdout, and pipe both to tee. This will keep the console output straight, but then there is no way to separate the stderr output from the stdout.

The only way to reliably get your result is to modify your source to write each error message to two streams right from the get go. But that is not possible when your source is out of your control.