In Bash (and other shell languages), how to preserve space escape characters in parameters that get passed through

Question

I have a script which when called directly, works correctly with the following combination of parameters:

--path=/path/to/a\ folder\ with\ spaces
--path='/path/to/a folder with spaces'
--path="/path/to/a folder with spaces"

but I am having difficulty passing parameters from a helper script to it.

I have tried numerous things but in each case it is as if my script receives the following: --path=/path/to/a folder with spaces which confuses my script and it is as if /path/to/a is the path (i.e. argument of --path) and folder with spaces are additional arguments.

As a minimal example, none of the following produces the desired result (save the below to a desired working directory as ./so_example.sh and chmod +x ./so_example.sh):

#!/usr/bin/env bash
set -x

printf "%s\n" "$@"

echo "$@"

echo -e "$@"

printf "%s\n" $@

echo $@

echo -e $@

When calling it in each of the following ways:

./so_example.sh --path=/path/to/a\ folder\ with\ spaces
./so_example.sh --path='/path/to/a folder with spaces'
./so_example.sh --path="/path/to/a folder with spaces"

I get the following output:

+ printf '%s\n' '--path=/path/to/a folder with spaces'
--path=/path/to/a folder with spaces
+ echo '--path=/path/to/a folder with spaces'
--path=/path/to/a folder with spaces
+ echo -e '--path=/path/to/a folder with spaces'
--path=/path/to/a folder with spaces
+ printf '%s\n' --path=/path/to/a folder with spaces
--path=/path/to/a
folder
with
spaces
+ echo --path=/path/to/a folder with spaces
--path=/path/to/a folder with spaces
+ echo -e --path=/path/to/a folder with spaces
--path=/path/to/a folder with spaces

I hope to get one of the following output for the correct solution:

--path=/path/to/a\ folder\ with\ spaces`
--path='/path/to/a folder with spaces'`
--path="/path/to/a folder with spaces"`

I did spend some time looking for answers on Stackoverflow. My search terms must have been inadequate, because only now I found https://stackoverflow.com/questions/21820240/preserving-escapes-in-bash-arguments which I think describes the problem I am facing. — dnk8n, Apr 19 '19 at 10:23
Put double quotes around `"$@"` everywhere, but you might find `"$*"` more appropriate, see https://unix.stackexchange.com/questions/41571/what-is-the-difference-between-and — cdarke, Apr 19 '19 at 10:32
I do demonstrate the use of double quotes in the question and that does not work. `printf "%q\n" "$@"` does however print it in a suitable way though. My real problem must be unrelated to this however but now I have a better way to debug at least. — dnk8n, Apr 19 '19 at 10:43
I'm not sure what the question is; if you quote parameter expansion, the given value is preserved. The quoting used to *create* the value, though, is not. (Because your script never *sees* that string; it's evaluated by the *caller* before your script is invoked.) If you are having problems passing such arguments to helper scripts, you are probably using regular variables where you should be using arrays. Other languages don't have similar problems, because they don't rely on parameter expansion. — chepner, Apr 19 '19 at 11:24
I think I found a problem in the wrong area of my script. Apologies. I have slightly better way of debugging exactly what the inner script sees now. — dnk8n, Apr 19 '19 at 11:45
Your code is correct. You probably misunderstand the output from `set -x`, or expect the wrong things. — tripleee, Apr 19 '19 at 11:46
The problem is not at all what I thought. I have created a more appropriate question which focuses on the problem - https://stackoverflow.com/questions/55762806 — dnk8n, Apr 19 '19 at 13:45

alecxs · Answer 1 · 2019-04-19T14:41:13.793

you have to explicitly use "$1" or iterate over "$@" in single processing. if you need to preserve spaces of "$@" within a single var, use string manipulation to mask with non-breaking spaces.

the following example will work with --path="PATTERN" and --path "PATTERN". it removes all paths from "$@" and move it in a single var $paths (in case "$@" contains other parameters you want to keep)

# remove options from positional parameters and read arguments
i=1
while [ $i -le $# ]
  do
    case "$1" in
      --path=*)
        test -a "$1"
        if [ $? != 0 ]
          then
            # allow nontrivial names in --path=PATTERN
            # (space, tab, newline, linefeed, formfeed, vertical tab)
            paths="${paths} ${1//[[:space:]]/[[:space:]]}"
            i=$(($i-1))
          else
            set -- "$@" "$1"
        fi
      ;;
      --path)
        test -a "$1"
        if [ $? != 0 ]
          then
            # allow nontrivial names in --path PATTERN
            # (space, tab, newline, linefeed, formfeed, vertical tab)
            paths="${paths} $1=${2//[[:space:]]/[[:space:]]}"
            shift
            i=$(($i-1))
          else
            set -- "$@" "$1"
        fi
      ;;
       *)
        set -- "$@" "$1"
      ;;
    esac
    shift
    i=$(($i+1))
done

# you can now pass the arguments to your command. the [[:space:]] will
# recognized as wildcard and will match all names containing spaces
<command> $paths

# however, if you want to display names (or if these paths not exist),
# you can convert it back (only spaces unfortunately)
for path in $paths
  do
    # revert back replacements [[:space:]] -> ' '
    path="${path//\[\[:space:]]/ }"

    # print everything before separator: ${VAR%%=*}
    # print the separator: (=)
    # print double quote: (\")
    # print everything after separator: ${VAR#*=}
    # print double quote: (\")

#    echo -e " ${path%%=*} = \" ${path#*=} \" "
    echo -e "${path%%=*}=\"${path#*=}\""
done

# other arguments are still available in positional parameters
for rest in "$@"
  do
    echo -e $rest
done

score 0 · Answer 2 · answered Apr 19 '19 at 11:30

The problem is you expect the syntactic quotes to be passed to your script; they aren't. All three of

--path=/path/to/a\ folder\ with\ spaces
--path='/path/to/a folder with spaces'
--path="/path/to/a folder with spaces"

are evaluated by the caller to produce the exact same value (--path=/path/to/a folder with spaces) for the argument. Quoting the positional parameter in your script preserves those spaces, but your script has no way of knowing how that value was created by whoever called it. (It might have been created in a way that your script knows nothing about; consider a command like dash -c 'printf "%s\n" "$1"' _ $'\t' run from bash. dash gets a single tab character as its argument, but it wouldn't know anything about the $'...' syntax bash used to create it.)

In Bash (and other shell languages), how to preserve space escape characters in parameters that get passed through

2 Answers2