6

I have a long path name to a program I must run in a for /f loop, which includes a closing parenthesis ")", and from which I need to parse the output:

for /f "tokens=1" %%G in ('"C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe" list') do (echo Will do something with %%G)

...where 'list' is a parameter passed to my program. I get the error "'C:\Documents' is not recognized as an internal or external command, operable program or batch file."

I do know the problem is that the closing parenthesis in fact closes the "for" block, so the ending double quotes is not "seen", so the long path name is not enclosed within double quotes anymore. What I don't understand is why is this happening, since my path is enclosed within double quotes? I also tried the usebackq option:

for /f "usebackq tokens=1" %%G in (`"C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe" list`) do (echo Will do something with %%G)

...with no better results. I tried to escape like this "^)" or like this "^^)", nothing to do. Tried doubling the double quotes:

for /f "tokens=1" %%G in ('""C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe"" list') do (echo Will do something with %%G)

Still does not work.

I'm, furthermore, in fact using a variable that holds the path, which is not know in advance (built from %CD%), and EnableDelayedExpansion is activated. I tried the delayed expansion (which did fixed similar problems in other situations) to prevent the variable's expansion at read time and delay it at execution time:

setlocal EnableDelayedExpansion
set _var=%CD%\program.exe
@REM _var now contains C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe
for /f "tokens=1" %%G in ('"!_var!" list') do (echo %%G)
endlocal

Still doesn't work, don't understand why.

But, doubling the double quotes with delayed expansion in above code:

for /f "tokens=1" %%G in ('""!_var!"" list') do (echo %%G)

does work!... why... why having to do this??? What effect does it have? I don't understand. I also fear it may cause a problem in some specific circumstances...

Any idea?

Alec
  • 8,529
  • 8
  • 37
  • 63
philz
  • 63
  • 1
  • 4
  • Excuse me. You wrote you tried to escape the parentheses. Are you sure you used this way?: `for /f "tokens=1" %%G in ('"C:\Documents and Settings\myaccount\Desktop\Test_release ^(x86^)\program.exe" list') do (echo Will do something with %%G)` – Aacini Nov 19 '11 at 08:44
  • Yes, I tried it as mentioned right below my 2nd example, without any success. – philz Nov 19 '11 at 16:02

2 Answers2

5

Comments in the answers to this question indicate XP gives different behavior then newer Windows versions.

There is a known FOR /F bug in XP: http://www.dostips.com/forum/viewtopic.php?p=9062#p9062. But this problem is not related to that bug.

The actual problem stems from how FOR /F executes a command in the IN() clause. It uses CMD \C command (See How does the Windows Command Interpreter (CMD.EXE) parse scripts?)

You can observe this behavior by adding this line to Aacini's PROG.BAT example.

echo cmdcmdline=%cmdcmdline%

The next issue deals with how CMD deals with quotes that appear in the /C command, and why XP behaves differently than more recent Windows versions.

This command fails in XP, but succeeds in Vista and beyond:

for /f "delims=" %a in ('"test (this)\prog" args') do @echo %a

The command that FOR tries to execute (%cmdcmdline%) is the same in both versions (disregarding differences in %COMSPEC%):

C:\Windows\system32\cmd.exe /c "test (this)\prog" args

XP has a CMD design flaw in how it deals with the quotes. The flaw is even documented (but it is not recognized as a flaw). Vista and beyond partially fix the design flaw, but don't bother to correct the documentation.

Here is an excerpt from HELP CMD

If /C or /K is specified, then the remainder of the command line after
the switch is processed as a command line, where the following logic is
used to process quote (") characters:

    1.  If all of the following conditions are met, then quote characters
        on the command line are preserved:

        - no /S switch
        - exactly two quote characters
        - no special characters between the two quote characters,
          where special is one of: &<>()@^|
        - there are one or more whitespace characters between the
          two quote characters
        - the string between the two quote characters is the name
          of an executable file.

    2.  Otherwise, old behavior is to see if the first character is
        a quote character and if so, strip the leading character and
        remove the last quote character on the command line, preserving
        any text after the last quote character.

We want CMD to follow rule 1 so that quotes are preserved, but ( and ) violate the special character constraint on XP, so rule 2 is followed and the CMD tries to execute

test (this)\prog args

It should be fairly obvious why this fails!

I can't think of any reason why the special character constraint exists in rule 1. It defeats the whole purpose of what MS is attempting to do.

Apparently the design flaw is partially fixed in Vista and beyond, but they haven't updated the HELP documentation. Vista ignores the special characters ( and ) and processes the command using rule 1, the quotes are preserved, and everything works.

Update 2015-05-17: Unfortunately, Vista and beyond still treat @, ^, and & as special characters, even though they are valid characters within file names. Of course <, >, and | are treated as special characters, but they are not valid in a file name anyway. So for Vista and beyond, the documentation for rule 1 should read where special is one of: &<>@^|.

I've traced the behavior that everyone has documented, and it is all consistent with the above.

There is a way to execute the command on XP without using a delayed expansion variable, and it is compatible with Vista and beyond.

for /f "delims=" %a in ('^""test (this)\prog" args^"') do @echo %a

The opening and closing quotes are escaped so that the ) does not interfere with the FOR parser. The command that is executed for the IN() clause is

C:\Windows\system32\cmd.exe /c ""test (this)\prog" args"

Both XP and Vista follow rule 2 because there is more than two quotes, so CMD executes

"test (this)\prog" args

and everything works!

The rest of this answer is outdated but preserved to give context to existing comments.


Your very 1st code example should work; it cannot (make that should not) give the error message you describe. The error message breaks off the path at the first space, which implies that the path was not quoted or escaped. But you are "sure" it was quoted.

The key to the problem is three pieces of information near the end of your post:

  1. You are actually using a variable with delayed expansion

  2. this doesn't work: for /f "tokens=1" %%G in ('"!_var!" list') do (echo %%G)

  3. this works: for /f "tokens=1" %%G in ('""!_var!"" list') do (echo %%G)

If the value of var is already quoted, you will get the behavior you are describing.

The value of your var must be "C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe", including the quotes.

To make this explanation more readable I will shorten the path to "test (this)\prog.exe"

"!var!" fails because it expands to ""test (this)\prog.exe"", which effectively unquotes the path. The string has three areas, two that are quoted, and one in the middle that is not:

"empty quoted area"unquoted path"empty quoted area"

""!var!"" works because it expands to """test (this)\prog.exe""" and the path is now quoted again. There are now five areas within the string:

"empty quoted area"empty unquoted area"quoted path"empty unquoted area"empty quoted area"

The simple answer as to how you should proceed:

If the value of var is already quoted, then simply use !var! Edit- that doesn't work on XP: "!var!" works on both

If the value of var is not quoted, then use "!var!" Edit- that doesn't work on XP: ""!var!"" works on both

Community
  • 1
  • 1
dbenham
  • 127,446
  • 28
  • 251
  • 390
  • Thanks dbenham. Right quoting was part of the basic validation I made. If I've posted here, it's because I also think it should work, but it does not. If you say this sould work: for /f "tokens=*" %%G in ('"D:\Test folder (this)\bin\program.exe"') do (echo Hi there) then why do I have this error: D:\Test folder (this)>for /F "tokens=*" %G in ('"D:\Test folder (this)\bin\program.exe"') do (echo Hi there ) 'D:\Test' is not recognized as an internal or external command, operable program or batch file. Thanks again for help! – philz Nov 21 '11 at 14:46
  • @philz ??? The statement works for me. What OS are you using? I've been testing on Vista. – dbenham Nov 21 '11 at 15:36
  • @philz Based on your comments under Aacini's answer, It sounds like you agree with my answer. If so, please check off this answer as the solution. It's too bad your comment is there and not here. I think I started things off on the wrong foot with where I placed some of my comments. – dbenham Nov 22 '11 at 15:48
  • _Copy of my comment under Aacini's one:_ Whoa!! You're my man! Right on! I had already managed to detect the OS version and execute one syntax on XP (using doubled double quotes and delayed expansion) and another one on Vista and up ("standard" syntax), but I will use your, since it's more elegant and applies to any OS versions. Two thumbs up on that one! Thanks again! – philz Nov 22 '11 at 16:39
1

I made some tests to find why the direct method you used at first didn't work. I first created a directory named test (this) and inside it I created a batch file named prog.bat:

>dir /b
test (this)
>dir /b "test (this)"
prog.bat
>type "test (this)\prog.bat"
@echo off
echo/One - Line one of prog
echo/Two - Line two of prog
echo/Three - Line three of prog

Then I first tried to access the contents of such file with for /f. Note that useback option is required because the file name include spaces and must be enclosed in quotes:

>for /f "usebackq tokens=1" %G in ("test (this)\prog.bat") do @echo %G
@echo
echo/One
echo/Two
echo/Three

Previous result prove that the pathname was correctly given. Now, to execute the Batch file instead of read it, just enclose its name in back quotes, right?

>for /f "usebackq tokens=1" %G in (`"test (this)\prog.bat"`) do @echo %G
'test' is not recognized as an internal or external command,
operable program or batch file.

Then, because the message said that test is the name of the command not found, I created a Batch file named test.bat:

>type test.bat
@echo off
echo Test.batFileLine1
echo Test.batFileLine2
echo Test.batFileLine2

>for /f "usebackq tokens=1" %G in (`"test (this)\prog.bat"`) do @echo %G
Test.batFileLine1
Test.batFileLine2
Test.batFileLine2

Aha! Previous result show that in this case the quotes are ignored and just test (this)\prog.bat is executed, right?

>for /f "usebackq tokens=1" %G in (`test (this)\prog.bat`) do @echo %G
\prog.bat`) was unexpected at this time.

Now what? Well, the problem NOW is related to the parentheses:

>for /f "usebackq tokens=1" %G in (`test ^(this^)\prog.bat`) do @echo %G
Test.batFileLine1
Test.batFileLine2
Test.batFileLine2

My conclusion is that the for /f command have an error when both back-quotes and quotes are used in combination with "usebackq" option and the filename have spaces, and that this bug is omitted if a delayed variable expansion is used.

Aacini
  • 65,180
  • 12
  • 72
  • 108
  • Aacini, to be noted that your very last results shows the content of your 2nd batch file, named "test.bat". In fact, your last example using the escaping of the parenthesis don't work. Just delete your "test.bat", you'll see. – philz Nov 21 '11 at 15:28
  • I am really puzzled. I tested the execution of a batch file both with and without the USEBACKQ option, and everything worked as expected on my Vista machine. What OS are you testing on? – dbenham Nov 21 '11 at 15:40
  • I'm working on XP SP3 32bits. I just tested on Win 7 64bits, and it does work!! The exact and same script! I don't event need to use the usebackq option, just the straight double-quoted string/path inside single quotes, no escaping either. Is it an [undocumented] XP bug? The only way I'm able to make it work on XP is using delayed expansion, with doubled double quotes (withing single quotes), as in my original post. – philz Nov 21 '11 at 18:19
  • @philz I've updated my answer with references to a known XP FOR /F bug. I hope to run some tests on XP tonight (US east coast) – dbenham Nov 21 '11 at 19:37
  • @philz I ran some tests and I think I know what is happening, and how to fix it. Check out my edited answer. – dbenham Nov 22 '11 at 03:06
  • @dbenham Whoa!! You're my man! Right on! I had already managed to detect the OS version and execute one syntax on XP (using doubled double quotes and delayed expansion) and another one on Vista and up ("standard" syntax), but I will use your, since it's more elegant and applies to any OS versions. Two thumbs up on that one! Thanks again! – philz Nov 22 '11 at 14:00