Escaping Spaces in Windows

Question

I've got lots of experience in a Linux environment, but now I need to do something in Windows. I'm having some trouble figuring out how to correctly escape spaces in a windows command environment. The rules about how it parses the command are completely unclear.

For example, in a Linux environment, I can create the following script:

my command.sh

#!/bin/bash
echo "Called as '$0' with argument '$1'"

and execute it like so, with all possible ways of escaping spaces being equivalent.

[bradworkubuntu:~/command_test] my\ command.sh arg\ with\ spaces
Called as './my command.sh' with argument 'arg with spaces'
[bradworkubuntu:~/command_test] "my command.sh" arg\ with\ spaces
Called as './my command.sh' with argument 'arg with spaces'
[bradworkubuntu:~/command_test] my\ command.sh "arg with spaces"
Called as './my command.sh' with argument 'arg with spaces'
[bradworkubuntu:~/command_test] "my command.sh" "arg with spaces"
Called as './my command.sh' with argument 'arg with spaces'
[bradworkubuntu:~/command_test] /home/brichardson/command_test/my\ command.sh arg\ with\ spaces
Called as '/home/brichardson/command_test/my command.sh' with argument 'arg with spaces'
[bradworkubuntu:~/command_test] "/home/brichardson/command_test/my command.sh" arg\ with\ spaces
Called as '/home/brichardson/command_test/my command.sh' with argument 'arg with spaces'
[bradworkubuntu:~/command_test] "/home/brichardson/command_test/my command.sh" "arg with spaces"
Called as '/home/brichardson/command_test/my command.sh' with argument 'arg with spaces'

but when I try something similar on Windows, the results are totally inconsistent.

my command.bat

echo "Called as '%0' with argument '%1'"

executed like so:

C:\Users\brichardson\Documents\command_test>my^ command.bat arg^ with^ spaces
'my' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\brichardson\Documents\command_test>"my command.bat" arg^ with^ spaces

C:\Users\brichardson\Documents\command_test>echo "Called as '"my command.bat"' with argument 'arg'"

"Called as '"my command.bat"' with argument 'arg'"

C:\Users\brichardson\Documents\command_test>my^ command.bat "arg with spaces"
'my' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\brichardson\Documents\command_test>"my command.bat" "arg with spaces"

C:\Users\brichardson\Documents\command_test>echo "Called as '"my command.bat"' with argument '"arg w
ith spaces"'"
"Called as '"my command.bat"' with argument '"arg with spaces"'"

C:\Users\brichardson\Documents\command_test>\Users\brichardson\Documents\command_test\my^ command.ba
t arg^ with^ spaces
'\Users\brichardson\Documents\command_test\my' is not recognized as an internal or external command,

operable program or batch file.

C:\Users\brichardson\Documents\command_test>"\Users\brichardson\Documents\command_test\my command.ba
t" arg^ with^ spaces

C:\Users\brichardson\Documents\command_test>echo "Called as '"\Users\brichardson\Documents\command_t
est\my command.bat"' with argument 'arg'"
"Called as '"\Users\brichardson\Documents\command_test\my command.bat"' with argument 'arg'"

C:\Users\brichardson\Documents\command_test>"\Users\brichardson\Documents\command_test\my command.ba
t" "arg with spaces"

C:\Users\brichardson\Documents\command_test>echo "Called as '"\Users\brichardson\Documents\command_t
est\my command.bat"' with argument '"arg with spaces"'"
"Called as '"\Users\brichardson\Documents\command_test\my command.bat"' with argument '"arg with spa
ces"'"

C:\Users\brichardson\Documents\command_test>C:\Users\brichardson\Documents\command_test\my^ command.
bat arg^ with^ spaces

C:\Users\brichardson\Documents\command_test>echo "Called as 'C:\Users\brichardson\Documents\command_
test\my command.bat' with argument 'arg'"
"Called as 'C:\Users\brichardson\Documents\command_test\my command.bat' with argument 'arg'"

C:\Users\brichardson\Documents\command_test>C:\Users\brichardson\Documents\command_test\my^ command.
bat "arg with spaces"

C:\Users\brichardson\Documents\command_test>echo "Called as 'C:\Users\brichardson\Documents\command_
test\my command.bat' with argument '"arg with spaces"'"
"Called as 'C:\Users\brichardson\Documents\command_test\my command.bat' with argument '"arg with spa
ces"'"

C:\Users\brichardson\Documents\command_test>"C:\Users\brichardson\Documents\command_test\my command.
bat" "arg with spaces"

C:\Users\brichardson\Documents\command_test>echo "Called as '"C:\Users\brichardson\Documents\command
_test\my command.bat"' with argument '"arg with spaces"'"
"Called as '"C:\Users\brichardson\Documents\command_test\my command.bat"' with argument '"arg with s
paces"'"

Can anyone explain to me when Windows command prompt will or won't escape spaces? And what it will actually store in the environment when it does or doesn't?

Near as I can tell, it only escapes spaces in the command path, and only if the drive letter is included. And if you use quotes to deal with spaces, it includes them in whatever string gets passed to the command.

Answer 1

The most important thing to understand is that there are two separate stages of parsing taking place. The first stage is the command interpreter looking for special characters, e.g., input redirection. The second stage is the executable parsing whatever is left of the command line into separate arguments.

In this particular case you are running a batch file, so the second stage of parsing is handled by the batch processor. (Note that the batch processor isn't actually a separate process to the command interpreter, but for our purposes they are best thought of as conceptually distinct.)

So if you start with

myapp.exe two^ is ^> one, "I think" > test.txt

The command interpreter redirects the output and passes this command line to the executable:

myapp.exe two is > one, "I think"

If the executable uses the default parsing provided by the C runtime, that becomes an array

"myapp.exe", "two", "is", ">", "one,", "I think"

(The answers to this question go into a lot more detail as to how the command line is processed.)

The first reason this is relevant to your question is because ^ is only a special character during the first stage of parsing. Therefore, it escapes special characters such as > < and | but has no effect on the way the command line is split into separate arguments. This explains the behaviour of many of your test cases.

The second reason this is relevant is that the parsing rules for the second stage are entirely up to the executable, or in this case the batch processor, to decide. As it happens, the batch processor makes the choice to retain the quote marks when an argument contains spaces. (This is presumably so that filenames containing spaces will be handled correctly; most batch file arguments are filenames.)

Note: if you don't want the quote marks, use %~1 instead of just %1. This is documented in the context of the for command but also applies to command-line arguments.

It should also be noted that the default parsing provided by the C runtime library does not include the quotes as part of the argument.

... and I think that covers everything, except ...

The observation that

c:\path\my^ command 

runs my command rather than my was a surprise to me. I can't explain this. It is particularly bad news if my command is a C-based executable, because "command" will be doubled up, i.e., the executable will see it as the first argument.

I think this is a bug. I recommend you avoid doing this.