1

I am trying to copy all the content from a remote folder into my local machine, via bash script:

    #!/usr/bin/env bash

    REMOTE_SOURCE="/absolute/path/to/source/data"
    TARGET="/absolute/local/path/to/target"
    SSH="ssh myuser@myhost"

    cd $TARGET
    echo $PWD

    TRANSFER="$SSH -- 'cd $REMOTE_SOURCE; tar cz ./' | tar xz"

    echo $TRANSFER
    $TRANSFER

Running the script, with the transfer command stored inside a variable fails with:

bash: cd /absolute/path/to/source/data; tar cz ./: No such file or directory

While it works correctly when copying the output from echo $TRANSFER and running directly into the shell:

ssh myuser@myhost -- 'cd /absolute/path/to/source/data; tar cz ./' | tar xz

Note:

  • The remote folder of course exists, so the reported error is confusing and doesn't help me to understand what to fix.
  • I get the error also removing the tar cz ./ part and leaving just the cd command.

UPDATE:

Removing the '-quoting makes the commands work.

But how to pipe the ssh result into tar xz then?

Solution:

The script should quote as less as possible, avoiding too much variable expansion magic:

echo "$SSH -- \"cd $REMOTE_DATA; tar cz ./  \" | tar xz"
$SSH -- "cd $REMOTE_DATA; tar cz ./" | tar xz
Kamafeather
  • 8,663
  • 14
  • 69
  • 99
  • 1
    *stored inside a variable*? Add your script to your question. – Cyrus Apr 01 '20 at 04:44
  • script added. Hopefully is more clear now. – Kamafeather Apr 01 '20 at 05:11
  • 4
    See [BashFAQ #50](http://mywiki.wooledge.org/BashFAQ/050), and specifically [**5. I'm constructing a command based on information that is only known at run time**](http://mywiki.wooledge.org/BashFAQ/050#I.27m_constructing_a_command_based_on_information_that_is_only_known_at_run_time). – David C. Rankin Apr 01 '20 at 05:19
  • 1
    See: [Difference between single and double quotes in bash](http://stackoverflow.com/q/6697753/3776858) – Cyrus Apr 01 '20 at 05:51
  • I knew the difference but I was having the wrong expectation that, being placed inside a double-quote, the single-quotes weren't having any special meaning (but probably I should have escaped them with `\'`) and that it would have interpolated the strings anyway. Also, I am now wondering ***when** does the interpolation happen: when assigning the `TRANSFER` variable or when using it?* – Kamafeather Apr 01 '20 at 09:12
  • Parsing of quotes happens at the very beginning of command execution, **before** parameter expansion happens. So, when you have `$TRANSFER` as a command, there are no quotes, *and then quotes from the expansion will never be parsed, because that phase of parsing is already over*. – Charles Duffy Apr 01 '20 at 22:22
  • Keep [BashFAQ #48](http://mywiki.wooledge.org/BashFAQ/048) in mind when evaluating proposed answers on the linked questions. `eval` introduces serious security vulnerabilities. Use the practices from [BashFAQ #50](http://mywiki.wooledge.org/BashFAQ/050) instead. – Charles Duffy Apr 01 '20 at 22:25

1 Answers1

0

Bash supports arrays and you're running into quoting issues (and running entirely unquoted leaves you open to accidental mayhem).

Let's update this to use arrays:

#!/usr/bin/env bash

REMOTE_SOURCE="/absolute/path/to/source/data"
TARGET="/absolute/local/path/to/target"
SSH="myuser@myhost"

cd "$TARGET"
echo "$PWD"

TRANSFER=(ssh "$SSH" -- "cd $REMOTE_SOURCE; tar cz ./")
AFTER=(tar xz)

echo "${TRANSFER[@]} | ${AFTER[@]}"
"${TRANSFER[@]}" | "${AFTER[@]}"

This needed an extra variable because you can't put the | in there without it being quoted, which would prevent it from acting as you desire.

Adam Katz
  • 14,455
  • 5
  • 68
  • 83
  • *grumbles a bit about using variables in reserved all-caps namespace* – Charles Duffy Apr 01 '20 at 22:23
  • This code would also benefit from changing `;` to `&&` so the `tar` doesn't run in the wrong directory if the `cd` it follows fails. And the second-to-last-line `echo` is by no means guaranteed to write output that does the same thing that the compound-command pipeline that follows it does; you'd need to use something that generates `eval`-safe quoting to get that guarantee. Personally, I'd leave it out in favor of using `set -x` to debug. – Charles Duffy Apr 01 '20 at 22:28
  • @CharlesDuffy – I agree. I thought I was changing enough and wanted to preserve as much as possible from the original question. – Adam Katz Apr 02 '20 at 14:14