7

Suppose you run test.bat "blabla,blabla,^>blabla", "blaby"

test.bat implementation:

@SETLOCAL
@ECHO OFF
SET list=%~1
ECHO "LIST: %list%"
ECHO "ARG 1: %~1"
ECHO "ARG 2: %~2"
@ENDLOCAL   
@GOTO :EOF

Output is as expected:

"LIST: blabla,blabla,>blabla"
"ARG 1: blabla,blabla,^>blabla"
"ARG 2: blaby"

But what if you make test.bat a function inside a batch file:

@SETLOCAL
CALL :TEST "blabla,blabla,^>blabla", "blaby"
@ENDLOCAL
@GOTO :EOF

:TEST
@SETLOCAL
@ECHO OFF
SET list=%~1
ECHO "LIST: %list%"
ECHO "ARG 1: %~1"
ECHO "ARG 2: %~2"
@ENDLOCAL   
@GOTO :EOF

After running it the output is:

"LIST: blabla,blabla,^"
"ARG 1: blabla,blabla,^^>blabla"
"ARG 2: blaby"

Huh?

  1. Where did blabla go in the LIST?
  2. ARG 1 has ^^? Why?

Can someone explain how special characters behave differently in function arguments as opposed to command line arguments?

Davor Josipovic
  • 5,296
  • 1
  • 39
  • 57

2 Answers2

10

You can get the same result with your 1st batch script simply by using:

call test.bat "blabla,blabla,^>blabla", "blaby"

Your problems stem from an unfortunate aspect of how batch processing parses CALL statements. It is described in phase 6 at How does the Windows Command Interpreter (CMD.EXE) parse scripts?.

Ouch - I thought I understood the caret doubling before, but obviously not. I've heavily edited the following discussion in response to jeb's comments.

The designers of CMD.EXE want a statement like call echo ^^ to give the same result as echo ^^. Both statements reduce ^^ to ^ in phase 2 where special characters are handled. But the CALL statement has to go through phase 1 and phase 2 a second time. So behind the scenes, when CMD.EXE recognizes the CALL statement in phase 6, it doubles the remaining caret back to ^^ and then the second round of phase 2 reduces it back to ^. Both statements echo a single caret to the screen.

Unfortunately CMD.EXE blindly doubles all carets, even if they are quoted. But a quoted caret is not treated as an escape, it is a literal. The carets are no longer consumed. Very unfortunate.

Running call test.bat "blabla,blabla,^>blabla", "blaby" becomes call test.bat "blabla,blabla,^^>blabla" "blaby" in phase 6 of the parser.

That easily explains why ARG 1 looks like it does in your output.

As far as where did blabla go?, that is just a bit trickier.

When your script executes SET list=%~1, the quotes are removed, the ^^ is treated as an escaped caret which reduces to ^, and the > is no longer escaped. So the output of your SET statement is redirected to a "blabla" file. Of course SET has no output so you should have a zero length "blabla" file on your hard drive.

EDIT - How to correctly pass the desired arguments using "late expansion"

In his answer, davor attempted to reverse the effect of the caret doubling within the called routine. But that is not reliable because you cannot know for sure how many times the carets may have been doubled. It is better if you let the caller adjust the call to compensate instead. It is tricky - you have to use what jeb called "late expansion"

Within a batch script you can define a variable that contains the desired argument string, and then delay the expansion until after carets are doubled by escaping the % with another %. You need to double the percents for each CALL in the statement.

@echo off
setlocal
set arg1="blabla,blabla,^>blabla"
call :TEST %%arg1%% "blaby"
echo(
call call :TEST %%%%arg1%%%% "blaby"
::unquoted test
exit /b

:TEST
setlocal
set list=%~1
echo "LIST: %list%"
echo "ARG 1: %~1"
echo "ARG 2: %~2"
exit /b

The above produces the desired result:

"LIST: blabla,blabla,>blabla"
"ARG 1: blabla,blabla,^>blabla"
"ARG 2: blaby"

"LIST: blabla,blabla,>blabla"
"ARG 1: blabla,blabla,^>blabla"
"ARG 2: blaby"

The expansion rules are different when run from the command line. It is impossible to escape a % from the command line. Instead you must add a caret within the percents that prevents the expansion phase from recognizing the name on the 1st pass, and then when the caret is stripped in phase 2, the 2nd pass of expansion properly expands the variable.

The following uses davor's original TEST.BAT

C:\test>test.bat "blabla,blabla,^>blabla" "blaby"
"LIST: blabla,blabla,>blabla"
"ARG 1: blabla,blabla,^>blabla"
"ARG 2: blaby"

C:\test>set arg1="blabla,blabla,^>blabla"

C:\test>test.bat %arg1% "blaby"
"LIST: blabla,blabla,>blabla"
"ARG 1: blabla,blabla,^>blabla"
"ARG 2: blaby"

C:\test>call test.bat %^arg1% "blaby"
"LIST: blabla,blabla,>blabla"
"ARG 1: blabla,blabla,^>blabla"
"ARG 2: blaby"

C:\test>set arg2=%^arg1%

C:\test>call call test.bat %^arg2% "blaby"
"LIST: blabla,blabla,>blabla"
"ARG 1: blabla,blabla,^>blabla"
"ARG 2: blaby"

An alternative to escaping - pass values by reference!

All in all, the escape rules are ridiculously complicated. That is why advanced batch scripting often passes string values by reference instead of as literals. The desired string is placed in a variable and then the name of the variable is passed as an argument. Delayed expansion is used to get the exact string without fear of corruption due to special characters or CALL caret doubling or percent stripping.

Here is a simple test.bat that demonstrates the concept

@echo off
setlocal enableDelayedExpansion
set "var1=!%~1!"
echo var1=!var1!

call :test var1
exit /b

:test
set "var2=!%~1!"
echo var2=!var2!

And here is a demonstration of how it works.

C:\test>set complicatedString="This & that ^" ^& the other thing ^^ is 100% difficult to escape

C:\test>set complicatedString
complicatedString="This & that ^" & the other thing ^ is 100% difficult to escape

C:\test>test.bat complicatedString
var1="This & that ^" & the other thing ^ is 100% difficult to escape
var2="This & that ^" & the other thing ^ is 100% difficult to escape

C:\test>call test.bat complicatedString
var1="This & that ^" & the other thing ^ is 100% difficult to escape
var2="This & that ^" & the other thing ^ is 100% difficult to escape

C:\test>call call test.bat complicatedString
var1="This & that ^" & the other thing ^ is 100% difficult to escape
var2="This & that ^" & the other thing ^ is 100% difficult to escape
Community
  • 1
  • 1
dbenham
  • 127,446
  • 28
  • 251
  • 390
  • +1 even the part of 'call echo this ^& that' is completely wrong :-) – jeb Sep 30 '12 at 11:22
  • 2
    It does not echo anything, as in phase 2, the caret is consumed. So later there isn't any caret that can be doubled, so there is only a nake ampersand and the complete 'call echo' fails. It's not possible to escape anything literaly in a 'call echo' only with late expansion. – jeb Sep 30 '12 at 12:06
  • @jeb - thanks - I think I've fixed my discussion of the reason why CMD.EXE doubles the carets. Another terrible MS design. Life would be so much easier if MS let us explicitly double escape as needed when using CALL. – dbenham Sep 30 '12 at 13:59
  • Thank you for the link and explanation dbenham. I find this very complex, probably because I still don’t get the basics of batch processing. No wonder I had so much trouble the last few days with these argument passings. – Davor Josipovic Sep 30 '12 at 15:29
  • 1
    @dbenham Quite some useful tips you bring up. Pity I can only do +1. Thank you. – Davor Josipovic Oct 01 '12 at 07:05
  • @dbenham a million thanks--the ByRef solution solves my problem! – Mark Berry Dec 03 '15 at 19:26
0

After some testing and the answer of dbenham, it apears one would need to anticipate the double caret and replace it back with single caret:

@SETLOCAL
CALL :TEST "blabla,blabla,^>blabla", "blaby"
@ENDLOCAL
@GOTO :EOF

:TEST
@SETLOCAL
@ECHO OFF
SET "list=%~1" & REM CHANGED
SET "list=%list:^^=^%" & REM ADDED
ECHO "LIST: %list%"
ECHO "ARG 1: %~1"
ECHO "ARG 2: %~2"
@ENDLOCAL   
@GOTO :EOF

The output then is:

"LIST: blabla,blabla,^>blabla"
"ARG 1: blabla,blabla,^^>blabla"
"ARG 2: blaby"

Note also the strange thing: in SET "list=%list:^^=^%" the ^^ between %% is taken as two characters, not as the escaped ^.

Davor Josipovic
  • 5,296
  • 1
  • 39
  • 57
  • 1
    That works as long as you know how your routine will be called. It will break if it is called using `call call :test ...`. Unlikely, but there are rare circumstances where double call is wanted. More problematic is your original "TEST.BAT". It may be executed using `TEST ...` or `CALL TEST ...`. Your script has no way to tell if the carets need to be fixed. I will update my answer to show how to get your desired result by modifying the CALL statement using the "late expansion" technique that jeb cryptically referenced. – dbenham Sep 30 '12 at 15:46