3

I am trying to understand more about how the Windows CMD parser works. I have been reading several posts about the CMD parser, including this one, but I can't seem to figure out why both carets (^) in the following code are stripped when delayed expansion is DISABLED:

@echo off

setlocal disabledelayedexpansion

set $test_var=This is text with escaped delayed expansion syntax - ^^!$var1^^! and ^^!$var2^^!

echo $test_var = %$test_var%

echo.
pause

I expected the result of running the code to produce the following output:

$test_var = This is text with escaped delayed expansion syntax - ^!$var1^! and ^!$var2^!

Instead, ALL carets are removed:

$test_var = This is text with escaped delayed expansion syntax - !$var1! and !$var2!

From reading the post about the parser, it is my understanding that Phase 2 removes special characters, which includes the caret (^) escape character. From my reading it seems that only one (1) caret character should be removed. Why are both carets removed?

Thanks For Your Help!

Community
  • 1
  • 1
Bill Vallance
  • 471
  • 3
  • 19
  • 1
    The first caret becomes removed in the `set` command line, the second one when echoing the normally (`%`-)expanded variable, because this happens before phase 2... – aschipfl Jul 09 '19 at 15:55
  • @aschipfl, let me make sure I understand what you are saying. The `set` command removes the 1st caret when it is processed by Phase 2. The `echo` command removes the 2nd caret when it is processed by Phase 2. Is this correct? – Bill Vallance Jul 09 '19 at 16:13
  • Almost exactly, but not the commands like `set` ot `echo` remove anything, but it is the command interpreter; but yes, it is always phase 2 in this situation... – aschipfl Jul 09 '19 at 16:28
  • So, removal of the carets is a function of when the carets get processed by the command interpreter. In my code, the carets get processed twice - the 1st time in the line of code with the `set` command and the 2nd time in the line of code with the `echo` command. Is this correct? – Bill Vallance Jul 09 '19 at 16:38
  • Thanks, everyone, for the great answers and tips. They are VERY helpful. @aschipfl, would you mind answering the question so that you get the credit? – Bill Vallance Jul 09 '19 at 18:09
  • 1
    Additional for debugging batch files: It's a good technic to use `set ` to display the *real* variable content, because it doesn't change the content, like `echo %varname%` would do – jeb Jul 10 '19 at 07:16
  • @jeb, GREAT TIP! I always wondered why I sometimes see `set` being used to echo variable content. Now I know why! – Bill Vallance Jul 10 '19 at 14:34

1 Answers1

3

The key to the answer is the fact that normal (%-)expansion (phase 1) occurs before recognition of special characters (in phase 2) — refer to the accepted answer to: How does the Windows Command Interpreter (CMD.EXE) parse scripts?

The first carets (^) become removed by the command interpreter in the line

set $test_var=This is text with escaped delayed expansion syntax - ^^!$var1^^! and ^^!$var2^^!

when phase 2 is done, so every instance of ^^ becomes one literal ^. You can prove this when you change @echo off to @echo on, so each parsed command line becomes echoed before being executed, or when you place the following command in the next line (thanks to user jeb for the hint):

rem // This displays the actual content of the variable:
set $test_var

In your next line

echo $test_var = %$test_var%

removal of the remaining carets happens, because – as already said – %-expansion (phase 1) happens first, resulting in a command line like

echo $test_var = This is text with escaped delayed expansion syntax - ^!$var1^! and ^!$var2^!

and then phase 2 follows, where the remaining carets are recognised and removed, resulting in this final text

$test_var = This is text with escaped delayed expansion syntax - !$var1! and !$var2!

You can protect the carets (as well as any other special characters) in the set command line when using the quoted syntax, like this:

set "$test_var=This is text with escaped delayed expansion syntax - ^!$var1^! and ^!$var2^!"

So you can save one level of escaping. However, this only works when the command extensions are enabled, but this is the default setting of the command interpreter anyway.

For the echo command line however, you cannot use such a method, because quotation marks became returned too.

I spotted that you used echo. to output an empty line. You should better use echo/ or echo( (refer to the external resource ECHO. FAILS to give text or blank line - Instead use ECHO/ to find out why).

By the way, your displayed text does not match the actual situation, because you have got delayed expansion disabled.

Mofi
  • 46,139
  • 17
  • 80
  • 143
aschipfl
  • 33,626
  • 12
  • 54
  • 99
  • Thanks, @aschipfl, for taking the time to write the answer to my question. This explains perfectly what is happening. Your last line of text "... your displayed text does not match the actual situation ..." is because the point of this exercise was to understand how to keep the exclamation marks so that I could use this code in a FOR /F loop that occurrs later in my .CMD file code. – Bill Vallance Jul 10 '19 at 13:18
  • You're welcome! I just wanted to mention the last sentence, because you might have seen something like `echo Hello world^^!` to display `Hello world!`, but this is a different situation, because the second `^` disappears during delayed expansion (phase 5) then; with your unquoted `set` syntax and delayed expansion enabled you'd need to escape every `!` like `^^^^^^^^^^!` (with the quoted `set` syntax it's `^^^^^!`)... – aschipfl Jul 10 '19 at 13:30