Batch set output and error of a command to separate variables

Question

In windows 7 batch (cmd.exe command-line), I am trying to redirect the standard output (stdout) and standard error (stderr) of a command to separate variables (so the 1st variables is set to the output, and the 2nd variable is set to the error (if any)) without using any temporary files. I have tried and tried with no success at this.

So, what would be a working way to set the output and error of a command to separate variables?

Answer 1

You could go for two nested for /F loops, where the inner one captures the standard output and the outer one captures the redirected error. Since the inner one instances a new cmd process, the captured text cannot just be assigned to a variable, because it will be lost after execution finishes. Rather I precede every line with | and just echo it to the standard output. The outer loop detects the leading | and separates the lines accordingly:

@echo off
setlocal EnableExtensions EnableDelayedExpansion
set "STDOUT="
set "STDERR="
(set LF=^
%=empty line=%
)
for /F "delims=" %%E in ('
    2^>^&1 ^(^
        for /F "delims=" %%O in ^('^
            command_line^
        '^) do @^(^
            echo ^^^|%%O^
        ^)^
    ^)
') do (
    set "LINE=%%E"
    if "!LINE:~,1!"=="|" (
        set "STDOUT=!STDOUT!!LINE:~1!!LF!"
    ) else (
        set "STDERR=!STDERR!!LINE!!LF!"
    )
)
echo ** STDOUT **!LF!!STDOUT!
echo ** STDERR **!LF!!STDERR!
endlocal
exit /B

The following restrictions apply to the code:

• empty lines are ignored;
• lines that begin with a semicolon ; are ignored;
• exclamation marks ! are lost, because delayed environment variable expansion is enabled;
• lines that begin with a pipe character | may be assigned wrongly;
• the overall size of data must not exceed 8190 bytes;

All of those limitations are true for both standard output and standard error.

---

Edit:

Here is an improved variant of the above code. The issues concerning empty lines and lines beginning with a semicolon ; are resolved, the other restrictions still remain:

@echo off
setlocal EnableExtensions EnableDelayedExpansion
set "STDOUT="
set "STDERR="
(set LF=^
%=empty line=%
)
for /F "delims=" %%E in ('
    2^>^&1 ^(^
        for /F "delims=" %%O in ^('^
            command_line ^^^^^^^| findstr /N /R "^"^
        '^) do @^(^
            echo ^^^^^^^|%%O^
        ^)^
    ^) ^| findstr /N /R "^"
') do (
    set "LINE=%%E"
    set "LINE=!LINE:*:=!"
    if "!LINE:~,1!"=="|" (
        set "STDOUT=!STDOUT!!LINE:*:=!!LF!"
    ) else (
        set "STDERR=!STDERR!!LINE!!LF!"
    )
)
echo ** STDOUT **!LF!!STDOUT!
echo ** STDERR **!LF!!STDERR!
endlocal
exit /B

The findstr command is used to precede every single line with a line number plus :, so no line appears empty to for /F; this prefix is removed later on, of course. This change also solves the ; issue implicitly.

Because of the nested piping into findstr, multiple escaping is required to hide the | character as long as its piping function is actually needed.

Answer 2

First off, batch does not have a simple method to capture multi-line output like unix shell scripting. You can use FOR /F to build a multi-line value line by line, but the total length is limited to < 8191 bytes, and the syntax is awkward. Or you could use FOR /F to capture multiple lines in a simulated array of variables.

With regard to your question, there is no way to independently capture both stdout and stderr without using at least one temporary file. EDIT: WRONG, aschipfl found a way. However, the temp file is faster, and a whole lot simpler.

Here is a simple demonstration that uses a file to capture stderr. I'm assuming you want to capture at most one line of stdout and/or stderr.

for /f "delims=" %%A in ('yourCommand 2^>err.log`) do set "out=%%A"
<err.log set /p "err="
del err.log

Here is a more complex example that captures an array of lines for both stdout and stderr. Here I assume that none of the lines of output begin with :. The FINDSTR prefixes each line with a line number followed by a :, and the FOR /F parses out the line number to be used as an "array" index, as well as the value after the :.

@echo off
setlocal disableDelayedExpansion
set /a out.cnt=err.cnt=0
for /f "delims=: tokens=1*" %%A in ('yourCommand 2^>err.log ^| findstr /n "^"') do (
  set "out.%%A=%%B"
  set "out.cnt=%%A"
)
for /f "delims=: tokens=1*" %%A in ('findstr /n "^" err.log') do (
  set "err.%%A=%%B"
  set "err.cnt=%%A"
)

:: Display the results
setlocal enableDelayedExpansion
echo ** STDOUT **
for /l %%N in (1 1 %out.cnt%) do echo(!out.%%N!
echo ** STDERR **
for /l %%N in (1 1 %err.cnt%) do echo(!err.%%N!

2nd Edit

Additional code is required if you want to properly handle output that begins with :.

@echo off
setlocal disableDelayedExpansion
set /a out.cnt=err.cnt=0
for /f "delims=" %%A in ('yourCommand 2^>err.log ^| findstr /n "^"') do for /f "delims=:" %%N in ("%%A") do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  for /f "delims=" %%B in (^""!ln:*:=!"^") do (
    endlocal
    set "out.%%N=%%~B"
    set "out.cnt=%%N"
  )
)
for /f "delims=" %%A in ('findstr /n "^" err.log') do for /f "delims=:" %%N in ("%%A") do (
  set "ln=%%A"
  setlocal enableDelayedExpansion
  for /f "delims=" %%B in (^""!ln:*:=!"^") do (
    endlocal
    set "err.%%N=%%~B"
    set "err.cnt=%%N"
  )
)

:: Display the results
setlocal enableDelayedExpansion
echo ** STDOUT **
for /l %%N in (1 1 %out.cnt%) do echo(!out.%%N!
echo ** STDERR **
for /l %%N in (1 1 %err.cnt%) do echo(!err.%%N!

---

Below I have adapted aschipfl's 2nd code that avoids using a temp file so that it preserves ! characters. The code just gets uglier and uglier ;-)

@echo off
setlocal disableDelayedExpansion
set "STDOUT="
SET "STDERR="
for /f "delims=" %%E in (
   '2^>^&1 (for /f "delims=" %%O in ('^
      yourCommand^
   ^^^^^^^| findstr /n /r "^"'^) do @(echo ^^^^^^^|%%O^)^) ^| findstr /n /r "^"'
) do (
   set "ln=%%E"
   setlocal enableDelayedExpansion
   set "ln=x!ln:*:=!"
   set "ln=!ln:\=\s!"
   if "!ln:~0,2!"=="x|" (
      set "ln=!ln:~0,-1!"
      for /f "delims=" %%A in (^""!STDOUT!"^") do for /f "delims=" %%B in (^""!ln:*:=!"^") do (
        endlocal
        set "STDOUT=%%~A%%~B\n"
      )
   ) else (
      for /f "delims=" %%A in (^""!STDERR!"^") do for /f "delims=" %%B in (^""!ln:~1!"^") do (
        endlocal
        set "STDERR=%%~A%%~B\n"
      )
   )
)
setlocal enableDelayedExpansion
for %%L in (^"^
%= empty line =%
^") do (
  if defined STDOUT (
    set "STDOUT=!STDOUT:\n=%%~L!"
    set "STDOUT=!STDOUT:\s=\!"
    set "STDOUT=!STDOUT:~0,-1!"
  )
  if defined stderr (
    set "STDERR=!STDERR:\n=%%~L!"
    set "STDERR=!STDERR:\s=\!"
    set "STDERR=!STDERR:~0,-1!"
  )
)

echo ** STDOUT **
echo(!STDOUT!
echo ** STDERR **
echo(!STDERR!
exit /b

It is a bit simpler if the result is stored in arrays instead of a pair of strings.

@echo off
setlocal disableDelayedExpansion
set /a out.cnt=err.cnt=1
for /f "delims=" %%E in (
   '2^>^&1 (for /f "delims=" %%O in ('^
      yourCommand^
   ^^^^^^^| findstr /n /r "^"'^) do @(echo ^^^^^^^|%%O^)^) ^| findstr /n /r "^"'
) do (
   set "ln=%%E"
   setlocal enableDelayedExpansion
   set "ln=x!ln:*:=!"
   if "!ln:~0,2!"=="x|" (
      set "ln=!ln:~0,-1!"
      for %%N in (!out.cnt!) do for /f "delims=" %%A in (^""!ln:*:=!"^") do (
        endlocal
        set "out.%%N=%%~A"
        set /a out.cnt+=1
      )
   ) else (
      for %%N in (!err.cnt!) do for /f "delims=" %%A in (^""!ln:~1!"^") do (
        endlocal
        set "err.%%N=%%~A"
        set /a err.cnt+=1
      )
   )
)
set /a out.cnt-=1, err.cnt-=1
setlocal enableDelayedExpansion

echo ** STDOUT **
for /l %%N in (1 1 %out.cnt%) do echo(!out.%%N!
echo ** STDERR **
for /l %%N in (1 1 %err.cnt%) do echo(!err.%%N!
exit /b

I know many people try to avoid temp files, but in this case I think it is counter productive. Tests have shown that temp files can be much faster than processing the result of a command with a FOR /F loop when the output is very large. And the temp file solution is so much simpler. So I would definitely use the temp file solution.

But finding a non-temp file solution is an interesting challenge. Kudos to aschipfl for working out the complicated escape sequences.

---

3rd (final?) Edit

At last, here is a solution that eliminates all restrictions, other than each captured line of output must be less than around 8180 bytes.

I could have put the entire code in one big loop, but then the escape sequences would have been a nightmare. Figuring out the escape sequences is much simpler when I break the code into smaller subroutines.

I capture the stdout and stderr for a bunch of ECHO commands found in the :test routine at the bottom.

::
:: Script to demonstrate how to run one or more commands
:: and capture stdout in one array and stderr in another array,
:: without using a temporary file.
::
:: The command(s) to run should be placed in the :test routine at the bottom.
::

@echo off
setlocal disableDelayedExpansion
if "%~1" equ ":out" goto :out
if "%~1" equ ":err" goto :err
if "%~1" equ ":test" goto :test

set /a out.cnt=err.cnt=0

:: Runs :err, which runs :out, which runs :test
:: stdout is captured in out array, and stderr in err array.
for /f "delims=. tokens=1*" %%A in ('^""%~f0" :err^"') do (
  for /f "delims=:" %%N in ("%%B") do (
    set "ln=%%B"
    setlocal enableDelayedExpansion
    for /f "delims=" %%L in (^""!ln:*:=!"^") do (
      endlocal
      set "%%A.%%N=%%~L"
      set "%%A.cnt=%%N"
    )
  )
)

:: Show results
setlocal enableDelayedExpansion
echo ** STDOUT **
for /l %%N in (1 1 %out.cnt%) do echo(!out.%%N!
echo(
echo ** STDERR **
for /l %%N in (1 1 %err.cnt%) do echo(!err.%%N!
exit /b


:err  :: 1) Run the :out code, which swaps stdout with stderr
      :: 2) Prefix stream 1 (stderr) output with err.###:  where ### = line number
      :: 3) Rredirect stream 2 (stdout) to combine with stream 1 (stderr)
2>&1 (for /f "delims=" %%A in ('^""%~f0" :out^|findstr /n "^"^"') do echo err.%%A)
exit /b


:out  :: 1) Run the :test code.
      :: 2) Prefix stream 1 (stdout) output with out.###:  where ### = line number
      :: 3) Swap stream 1 (stdout) with stream 2 (stderr)
3>&2 2>&1 1>&3 (for /f "delims=" %%A in ('^""%~f0" :test^|findstr /n "^"^"') do echo out.%%A)
exit /b


:test :: Place the command(s) to run in this routine
    echo STDOUT line 1 with empty line following
    echo(
>&2 echo STDERR line 1 with empty line following
>&2 echo(
    echo STDOUT line 3 with poison characters "(<^&|!%%>)" (^<^^^&^|!%%^>)
>&2 echo STDERR line 3 with poison characters "(<^&|!%%>)" (^<^^^&^|!%%^>)
    echo err.4:STDOUT line 4 spoofed as stderr - No problem!
>&2 echo out.4:STDERR line 4 spoofed as stdout - No problem!
    echo :STDOUT line 5 leading colon preserved
>&2 echo :STDERR line 5 leading colon preserved
    echo ;STDOUT line 6 default EOL of ; not a problem
>&2 echo ;STDERR line 6 default EOL of ; not a problem
exit /b

-- OUTPUT --

** STDOUT **
STDOUT line 1 with empty line following

STDOUT line 3 with poison characters "(<^&|!%>)" (<^&|!%>)
err.4:STDOUT line 4 spoofed as stderr - No problem!
:STDOUT line 5 leading colon preserved
;STDOUT line 6 default EOL of ; not a problem

** STDERR **
STDERR line 1 with empty line following

STDERR line 3 with poison characters "(<^&|!%>)" (<^&|!%>)
out.4:STDERR line 4 spoofed as stdout - No problem!
:STDERR line 5 leading colon preserved
;STDERR line 6 default EOL of ; not a problem

I still like the temp file solution much better ;-)

Answer 3

This solution works correctly as long as the lines sent to stdout does not start with the line number itself separated by colon.

@echo off
setlocal EnableDelayedExpansion

set /A out=0, err=1
for /F "tokens=1* delims=:" %%a in ('(theCommand 1^>^&2 2^>^&3 ^| findstr /N "^"^) 2^>^&1') do (
   if "%%a" equ "!err!" (
      set "stderr[!err!]=%%b"
      set /A err+=1
   ) else (
      set /A out+=1
      if "%%b" equ "" (
         set "stdout[!out!]=%%a"
      ) else (
         set "stdout[!out!]=%%a:%%b"
      )
   )
)
set /A err-=1

echo Lines sent to Stdout:
for /L %%i in (1,1,%out%) do echo !stdout[%%i]!
echo/
echo Lines sent to Stderr:
for /L %%i in (1,1,%err%) do echo !stderr[%%i]!

For example, if theCommand is this .bat file:

@echo off
echo Line one to stdout
echo Line one to stderr >&2
echo Line two to stderr >&2
echo Line two to stdout

... then this is the output:

Lines sent to Stdout:
Line one to stdout
Line two to stdout

Lines sent to Stderr:
Line one to stderr
Line two to stderr