Windows batch delayed expansion breaks running a pipe

Question

In a .bat script, I have a variable that contains & and needs to be passed to multiple commands, one of which is a pipe. Concept code which doesn't work:

set foo="1 & 2"
command1 --foo %foo% --bar
echo %foo% | command2 --stdin

After trying various solutions from here and here, I ended up quoting the value of the variable (as opposed to escaping and not quoting, or quoting the entire "var=value").

This works when passing an argument to commands; but when using echo in a pipe, the quotes are also passed to stdin:

setlocal enableExtensions enableDelayedExpansion
set foo="1 & 2"

echo %foo% | sort

Output (as expected):

"1 & 2"

Next, I added delayed expansion in order to remove the quotes, but now the pipe character becomes a literal character in the echo instead of running both commands:

setlocal enableExtensions enableDelayedExpansion
set foo="1 & 2"

echo !foo:"=! | sort

Output:

1 & 2 | sort

How do I convince the script to actually run the pipe instead of making it a literal string?

In case it matters, I'm running this in Windows 10.

Note: using sort above is simply an example of an arbitrary command, convenient because it takes stdin and prints it back, similar to the cat command in Linux.

@Compo yes, it outputs nothing for some mysterious reason. I also tried `cmd /c echo` and `for %%i in (!foo:"=!) echo %%i`. At this point I am looking to understand _why_ it keeps failing, so I don't lose my mind trying permutations. — Sir Athos, Mar 05 '23 at 00:31
I can't post my actual original code, but the "concept code" section is exactly what it's doing. There is a given string which needs to be passed by this script to multiple commands. At least one of those commands can only read it from stdin and at least one other command must have the string passed as a command line argument. The reason I picked `sort` for the example code is because it's a known command which takes stdin and does something predictable with it. I'm open to solutions other than the posted example, though the question is specifically "why does this approach not work?" — Sir Athos, Mar 05 '23 at 03:56
Try a look at [Why does delayed expansion fail when inside a piped block of code?](https://stackoverflow.com/q/8192318/463115) — jeb, Mar 05 '23 at 13:27

score 2 · Answer 1 · answered Mar 05 '23 at 12:59

As both sides of a pipe are executed in separate processes, you need to "double-escape" poison chars (and escaping one of the escape chars too to pass it to the second instance), which makes three escape chars:

setlocal enableExtensions enableDelayedExpansion
set "foo=1 & 2"
echo %foo:&=^^^&% | sort
rem or:
set "foo=1 ^^^& 2"
echo %foo% | sort

jeb · Accepted Answer · 2023-03-05T13:46:47.177

As Stephan said, both sides are executed in their own child processes, this results into multiple times of parsing the command.

You could solve it by escaping it multiple times, or you could build a command, that doesn't need to be escaped.

One way is to avoid expanding in the batch file itself and use only one escape level.
Like

set "foo=1 ^& 2"
echo %%foo%% | sort

The double percents will be parsed in the batch file and in the child process it becomes:

echo %foo%

Using late delayed expansion in the child process is even better, but is a bit tricky because the child process will start without delayed expansion enabled (default is 0x00/OFF in the registry key HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor\DelayedExpansion), you need to enabled it with an additional cmd.exe process.

set "foo=1 & 2"
(cmd /V:on /C "echo !foo!") | sort

The parenthesis avoid the delayed expansion in the batch file itself.

Or for more complex content, perhaps with IF or FOR-Loops involved in the left side, I would recommend to start a batch file itself in the child process.
In my example, I use the trampoline technique for that.

@echo off
REM *** Trampoline jump for function calls of the form ex. "C:\:function:\..\MyBatchFile.bat"
FOR /F "tokens=3 delims=:" %%L in ("%~0") DO goto :%%L

set "foo[0]=5 & 6"
set "foo[1]=7 ! >>"
set "foo[2]=8"
set "foo[3]=9%%"
"%~d0\:childProc:\..\%~pnx0" | sort
exit /b

:childProc
setlocal enabledelayedExpansion
for /L %%n in (0 1 3) do echo(!foo[%%n]!
exit /b

Yvil! I like it - but perhaps a `setlocal` after the `for/f` to avoid permasetting `foo*` — Magoo, Mar 05 '23 at 17:50

score 0 · Answer 3 · answered Mar 13 '23 at 06:31

After the helpful contributions of Stephan, jeb, and u/jcunews1, followed by painful wrangling of cmd, I've settled down on the following solution, which works both with passing the string as a command line argument, and piping to stdin, while using the same variable.

setlocal enableDelayedExpansion
set "FOO=This ^& that. Oh ^& the other thing^!"

some_app --foo-arg "!FOO!"
cmd /v:on /c "echo ^!FOO^!"| some_other_app_which_reads_stdin

REM Bonus round: background launch app that also requires stdin
start /B cmd /v:on /c cmd /v:on /c "echo ^^^!FOO^^^!"^| some_third_app

The special characters in the string are escaped with ^, and so are the exclamation marks in the pipe, because there is an additional level of cmd for each half of the pipe. Same principle applies for the last line, where yet another level of escaping has to be inserted since start runs a cmd which then runs a pipe.

I still don't understand why %FOO% fails to do the same thing and so this only works with delayed expansion, but it's a working solution.

Windows batch delayed expansion breaks running a pipe

3 Answers3