How to escape a variable in Bash when passing to a command-line argument

Question

I've got a Bash script (Cygwin) that uses some Windows paths with spaces in them. Consequently, I have escaped the space with a \ in my variable definition.

Everything within the script works fine. However, I need to pass this variable as an argument to a command-line executable. When I do that, my escaping gets messed up.

Sample non-functional script:

#!/bin/sh

# File paths
destinationPath="/cygdrive/c/Documents and Settings/Eric/My Documents/"
attachments="\n2013-12-12.pdf"
body="Test Body"
recipient="asdf@asdf.com"


# Prepare attachments
args=""
for file in $attachments ; do
    file=${file//[ \\n]/}
    touch $file
    mv $file "$destinationPath/$file"
    args="$args -a $destinationPath/$file"
done


# Send email
echo -e $body | email --from-addr nosender@mydomain.com --from-name "Automated CSS Downloader" --subject "Downloaded new file(s) \
  from CSS" $args eric@mydomain.com

Output from the script:

$ bash -x test.sh
+ destinationPath='/cygdrive/c/Documents and Settings/Eric/My Documents/'
+ attachments='\n2013-12-12.pdf'
+ body='Test Body'
+ recipient=asdf@asdf.com
+ args=
+ for file in '$attachments'
+ file=2013-12-12.pdf
+ touch 2013-12-12.pdf
+ mv 2013-12-12.pdf '/cygdrive/c/Documents and Settings/Eric/My Documents//2013-12-12.pdf'
mv: listing attributes of `2013-12-12.pdf': Invalid argument
+ args=' -a /cygdrive/c/Documents and Settings/Eric/My Documents//2013-12-12.pdf'
+ echo -e Test Body
+ email --from-addr nosender@mydomain.com --from-name 'Automated CSS Downloader' --subject 'Downloaded new file(s) from CSS' -a /cygdrive/c/Documents and Settings/Eric/My Documents//2013-12-12.pdf eric@mydomain.com
email: WARNING: Email address 'and' is invalid. Skipping...
email: FATAL: Could not open attachment: /cygdrive/c/Documents: No such file or directory

So, as you can see, the escaped space in the path is not being exported in the $args variable. I am assuming the error comes on the line "args=...". But I am not sure how to escape $destinationPath to ensure that the escaped characters are retained.

I've tried using double quotes (with no escaped space) in destinationPath, but to no avail. If I try to double quote $destinationPath in the args= line, then the output also gets all screwed up with a bunch of extra quoting.

How can I get this to work? I've tried playing around with the $IFS variable, but I don't really know what I'm doing with it and can't seem to get it working with that either, although I suspect the solution has something to do with $IFS.

Have you tried double-escaping the spaces? E.g. `Documents\\\ and\\\ ` — i Code 4 Food, Jun 23 '13 at 03:18
I had tried, but ran into a couple of problems: 1) my mv doesn't work as it is trying to move the file to a Documents\\ and \\ (ie: bash swallows one escape), and the expansion of $args in the email expression then becomes: -a '/cygdrive/c/Documents\' 'and\' 'Settings/Eric/My\' Documents/A.J/CSS//2013-06-21.pdf. Which doesn't work b/c of the multiple ' at every space. — Eric B., Jun 23 '13 at 03:26
Maybe you could take destinationPath into doublequoted and remove escaping spaces? Rest the same as you have — robson, Jun 23 '13 at 03:57
I've tried pretty much every combination I can think of. If I use doublequoted desinationPath (with no escaped strings), the email expansion comes out without quotes and then it breaks again. I have edited my original question with a fully functional sample script. — Eric B., Jun 23 '13 at 04:17
wild guess, but do try replacing `$destinationPath/$file` with `\"$destinationPath/$file\"`, see if it helps — doubleDown, Jun 23 '13 at 04:22
But now you removed also escaped double quoted from args where you put destination - leave them there — robson, Jun 23 '13 at 04:27

score 7 · Accepted Answer · edited Jul 31 '19 at 17:07

There's no good way of doing this without an array. (There's a not good way of doing it, using eval, but the array is simpler.)

The problem is that you want each file to end up as one argument to email, but you want to accumulate all those arguments into a Bash variable. There's just no way to do that: you can cause the Bash variable to be inserted as a single argument ("$arg") or you can cause it to be inserted as however many words it gets split into ($arg), but you can't get it to be split according to whether or not spaces were escaped when the variable was created, because Bash only remembers the string assigned to a variable, not the escape marks.

However, you can do it with an array, because you can make every filename exactly one array element, and you can get Bash to insert an array as one argument per element.

Here's how:

# File paths                                                                                                                              
destinationPath="/cygdrive/c/Documents and Settings/Eric/My Documents/"
attachments="\n2013-12-12.pdf"
body="Test Body"
recipient="asdf@asdf.com"


# Prepare attachments                                                                                                             
args=()
for file in $attachments ; do
    file=${file//[ \\n]/}
    touch $file
    mv $file "$destinationPath/$file"
    args+=(-a "$destinationPath/$file")
done

# Send email                                                                                                                      
echo -e $body |
  email --from-addr nosender@mydomain.com \
        --from-name "Automated CSS Downloader" \
        --subject "Downloaded new file(s) from CSS" \
        "${args[@]}" eric@mydomain.com

You might also want to make the attachments variable into an array; from the presence of the newline character, I'm assuming that you're actually setting it in some more complicated way, so I didn't change the code. But your current code won't work if the attachment name has a space in it (and I'm pretty sure that the newline will be eliminated by the parameter expansion in the for form, unless you've altered the value of $IFS, so file=${file//[ \\n]/} shouldn't be necessary.)

Thanks. Will give that a go. I didn't know/realize that Bash supported arrays. As for making attachments into an array, I would greatly prefer that, but I need to have it expanded into the body of my email, one line per attachment name. Will look into how to make that work easily. — Eric B., Jun 24 '13 at 02:00
Excellent. Worked like a charm! Need to learn how to better use arrays in Bash. Thanks! — Eric B., Jun 24 '13 at 02:57

score 1 · Answer 2 · answered Jun 23 '13 at 04:30

You need escaped double quoted marks when providing destinationPath in a string.

This should work:

#!/bin/sh                                                                                                                                 

# file paths                                                                                                                              
destinationPath="/cygdrive/c/Documents and Settings/Eric/My Documents/"
attachments="\n2013-12-12.pdf"
body="Test Body"
recipient="asdf@asdf.com"


        # prepare attachments                                                                                                             
        args=""
        for file in $attachments ; do
            file=${file//[ \\n]/}
            touch $file
            mv $file "$destinationPath/$file"
            #HERE YOU NEED ESCAPED DOUBLEQUOTED
            args="$args -a \"$destinationPath/$file\""
        done

        # send email                                                                                                                      
        echo -e $body | email --from-addr nosender@mydomain.com --from-name "Automated CSS Downloader" --subject "Downloaded new file(s) \
from CSS" $args eric@mydomain.com

This doesn't work, because bash does escape and quote interpretation before it substitutes variables, so if you put escapes/quotes/etc *in* variables they never take effect (by the time the variable's been expanded it's too late for them to do anything). Arrays (as in @rici's answer) are the best way to do this sort of thing. — Gordon Davisson, Jun 23 '13 at 06:00

How to escape a variable in Bash when passing to a command-line argument

2 Answers2

Linked