10

I'm trying to read a file containing filepaths line by line and scp the files to another server, but because of certain characters in the filenames like '(', ')', '&' etc. I need to escape the input:

input.txt:

/folder1/folderA/filename+(oh+how+nice,+parantheses)

script.sh:

#!/bin/sh

promote_to=random.server.com
dev_catalog=/folderX/
uat_catalog=/folderY/

while read line
do
uat_path=$(echo "$uat_catalog$line" | sed -e "s/(/\\\(/g" | sed -e "s/)/\\\)/g")
dev_path=$(echo "$dev_catalog$line" | sed -e "s/(/\\\(/g" | sed -e "s/)/\\\)/g")

scp $dev_path user@$promote_to:$uat_path
scp $dev_path".atr" user@$promote_to:$uat_path".atr"
done < "input.txt"

Output:

-bash: /folder1/folderA/filename+(oh+how+nice,+parantheses): No such file or directory
-bash: /folder1/folderA/filename+(oh+how+nice,+parantheses): No such file or directory
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
               [-l limit] [-o ssh_option] [-P port] [-S program]
               [[user@]host1:]file1 [...] [[user@]host2:]file2
ssh: random.server.com: Name or service not known
lost connection

Any kind of help is appreciated.

Andrei Ionita
  • 124
  • 1
  • 3
  • 8

3 Answers3

14

Part of the problem here is that the local and remote filenames are parsed differently: the local filename is used directly, so the only thing you need to do is enclose it in double-quotes (as in @Ignacio's answer), but the remote filename gets passed to a remote shell which runs it through another layer of parsing (quote and escape removal, etc). So, you want to add escapes to the remote path only. I've also taken the liberty of simplifying the sed command a little:

#!/bin/sh

promote_to=random.server.com
dev_catalog=/folderX/
uat_catalog=/folderY/

while read line
do
uat_path="$(echo "$uat_catalog$line" | sed -e 's/[()&]/\\&/g')"
dev_path="$dev_catalog$line"

scp "$dev_path" "user@$promote_to:$uat_path"
scp "$dev_path.atr" "user@$promote_to:$uat_path.atr"
done < "input.txt"

Note that the sed pattern I used, 's/[()&]/\\&/g', only escapes parentheses and ampersands; if your filenames contain any other shell metacharacters, be sure to add them to the character list in [].

Gordon Davisson
  • 118,432
  • 16
  • 123
  • 151
  • For those wondering on how to escape single quotes, see here: https://unix.stackexchange.com/questions/9746/how-to-replace-quotation-marks-in-a-file-with-sed – Josh Nov 23 '18 at 00:55
2

The other way around would be to NOT escape special chars but to quote the value in a format that can be re-used as an input instead (modern Bash is implied, e.g. version 5 and higher):

> VAR='/folder1/folderA/filename+(oh+how+nice,+parantheses)'
> echo $VAR
/folder1/folderA/filename+(oh+how+nice,+parantheses)
> echo ${VAR@Q}
'/folder1/folderA/filename+(oh+how+nice,+parantheses)'
George L. Yermulnik
  • 126
  • 3
0

You're trying to execute /folder1/folderA/filename+(oh+how+nice,+parantheses) as a command. You probably meant to do echo /folder1/folderA/filename+(oh+how+nice,+parantheses) | ... instead.

Edit: what @Ignacio said.

Eduardo Ivanec
  • 11,668
  • 2
  • 39
  • 42
  • that's right... missed the echo, but still doesn't work: "/folder1/folderA/filename+\\(oh+how+nice,+parantheses\\): No such file or directory" – Andrei Ionita Feb 28 '12 at 14:10
  • removing one backslash from each sed throws this however: "bash: -c: line 0: syntax error near unexpected token `(' bash: -c: line 0: `scp -t bash: -c: line 0: syntax error near unexpected token `(' bash: -c: line 0: `scp -t /folder1/folderA/filename+(oh+how+nice,+parantheses)' lost connection" – Andrei Ionita Feb 28 '12 at 14:14