1

I'm translating a GNU Makefile into MSBuild XML.

I have the following sed command (part of a larger command):

... | sed 's/^Q(.*)/"&"/' | ...

When I execute just that sed portion in Cygwin, it "works" in the sense that it doesn't error out.

However, after I've translated it to MSBuild XML - replaced the XML-sensitive symbols with ", &, ' - I get the error

sed: unsupported command '

I'm sure it's not an issue with XML escaping issues; the Visual Studio build log says

Task Parameter:Command="C:\Program Files\GNU ARM Eclipse\Build Tools\2.8-201611221915\bin\sed" 's/^Q(.*)/"&"/' (TaskId:21)

sed: unsupported command ' (TaskId:21)

The command ""C:\Program Files\GNU ARM Eclipse\Build Tools\2.8-201611221915\bin\sed" 's/^Q(.*)/"&"/' " exited with code 1. (TaskId:21)

The command was translated into the originally intended sed 's/^Q(.*)/"&"/'

However, there now appears to be an issue with cmd.exe.

With respect to cmd.exe, what part of that command doesn't it like?
Is it the single/double quote? The ampersand?

I'm using the table from here.

Community
  • 1
  • 1
Bob
  • 4,576
  • 7
  • 39
  • 107
  • No idea, since it appears to be the `sed` version you are using which is objecting to the command it is receiving. If you're sure it's `cmd` not co-operating, then `cmd` regards `^`, `)`,`&` as special characters. I'd try escaping each with `^` so `'s/^^Q(.*^)/"^&"/'` - `^` normally escapes problematic characters in `cmd` - except for `%` which is escaped by `%` – Magoo Oct 08 '17 at 19:16
  • @Magoo I put `'s/^Q(.*)/"&"/'` in a text file and called it like `sed -f sedCmd.txt` and I got the same error message. Does that narrow anything down? – Bob Oct 08 '17 at 19:20
  • This doesn't appear to have anything to do with `bash` – chepner Oct 08 '17 at 19:27
  • In the case of the caret, you may need to escape the escape character, `'s/^^^Q(.*)/"^&"/'`, I'm doubtful that the closing parenthesis requires escaping. – Compo Oct 08 '17 at 19:38
  • Perhaps you need `sed -e 's/...'` - escape characters required. Oh, `|`,`>` and `<` are other characters that may need to be escaped if used – Magoo Oct 08 '17 at 19:46
  • @Compo I tried that. Also tried escaping parentheses, tried escaping double quotes . . . sigh. Same error. – Bob Oct 08 '17 at 19:56
  • 1
    @Adrian when you write `command 'script'` for any given cmmand the `'`s are to delimit your script so your OS knows where it starts/ends to pass it to `command`. When you put the `script` in a file to be executed with `command -f`, the part you need to put in the file is just the script, not the `'`s that delimited it on the command line. So put `s/^Q(.*)/"&"/` in the script file instead of `'s/^Q(.*)/"&"/'` and try again. – Ed Morton Oct 08 '17 at 21:53
  • @EdMorton it works! But now how do I get it to work on the command line? – Bob Oct 08 '17 at 21:58
  • That is entirely a question about the nightmare of Windows quoting rules of which I have no knowledge other than that the usual advice I see is don't even try, just use script files. Good luck! – Ed Morton Oct 08 '17 at 21:59
  • @EdMorton I just removed the `'` and it works on the command line. Have I mentioned I HATE building on Windows? – Bob Oct 08 '17 at 22:00
  • @Adrian - Would you mind editing the question and add the final working command line? – lit Oct 08 '17 at 22:01
  • @lit I was told to not put the ANSWER in my question. So if someone wants to answer the question with the correct version, I’ll mark it as the answer: the working version is with the outer single quotes removed. – Bob Oct 08 '17 at 22:08

1 Answers1

2

The shell interpreters on Unix/Linux/Mac support at least three types of argument strings:

  1. None quoted argument strings for simple strings not containing a space character or a character with a special meaning for shell interpreter.
  2. " quoted argument strings for strings containing a space character or a character with a special meaning for shell interpreter not containing ".
  3. ' quoted argument strings for strings containing a space character or a character with a special meaning for shell interpreter not containing '.

The Windows command line interpreter supports only two types of argument strings.

  1. None quoted argument strings for simple strings not containing a space character or one of these characters: &()[]{}^=;!'+,`~|<>.
  2. " quoted argument strings for strings containing a space character or a character with a special meaning for Windows command interpreter not containing ".

For that reason a regular expression string as argument string containing " is problematic on Windows as there is no alternate solution for quoting the argument string as on *nix/Mac shells.

A solution for regular expressions in Perl syntax is using the hexadecimal notation \x22 for double quote character " in the regular expression.

So "s/^Q(.*)/\x22&\x22/" could work on Windows as well as on Linux instead of 's/^Q(.*)/"&"/'.

However, the command line argument strings are finally processed by the called application and it depends on the application and the used compiler how the application interprets the arguments. Therefore it depends always also on the application what finally works. See C code in this answer and the two different results on running this code compiled with three different compilers with a misquoted argument string. Another example for application dependent interpretation of argument string is explained in answer on How to set environment variables with spaces?

For that reason using just s/^Q(.*)/"&"/ without enclosing the regular expression argument string in the single quotes ' works for sed on Windows too.

Mofi
  • 46,139
  • 17
  • 80
  • 143