How can I escape an arbitrary string for use as a command line argument in Bash?

Question

I have a list of strings and I want to pass those strings as arguments in a single Bash command line call. For simple alphanumeric strings it suffices to just pass them verbatim:

> script.pl foo bar baz yes no
foo
bar
baz
yes
no

I understand that if an argument contains spaces or backslashes or double-quotes, I need to backslash-escape the double-quotes and backslashes, and then double-quote the argument.

> script.pl foo bar baz "\"yes\"\\\"no\""
foo
bar
baz
"yes"\"no"

But when an argument contains an exclamation mark, this happens:

> script.pl !foo
-bash: !foo: event not found

Double quoting doesn't work:

> script.pl "!foo"
-bash: !foo: event not found

Nor does backslash-escaping (notice how the literal backslash is present in the output):

> script.pl "\!foo"
\!foo

I don't know much about Bash yet but I know that there are other special characters which do similar things. What is the general procedure for safely escaping an arbitrary string for use as a command line argument in Bash? Let's assume the string can be of arbitrary length and contain arbitrary combinations of special characters. I would like an escape() subroutine that I can use as below (Perl example):

$cmd = join " ", map { escape($_); } @args;

Here are some more example strings which should be safely escaped by this function (I know some of these look Windows-like, that's deliberate):

yes
no
Hello, world      [string with a comma and space in it]
C:\Program Files\ [path with backslashes and a space in it]
"                 [i.e. a double-quote]
\                 [backslash]
\\                [two backslashes]
\\\               [three backslashes]
\\\\              [four backslashes]
\\\\\             [five backslashes]
"\                [double-quote, backslash]
"\T               [double-quote, backslash, T]
"\\T              [double-quote, backslash, backslash, T]
!1                
!A                
"!\/'"            [double-quote, exclamation, backslash, forward slash, apostrophe, double quote]
"Jeff's!"         [double-quote, J, e, f, f, apostrophe, s, exclamation, double quote]
$PATH             
%PATH%            
&                 
<>|&^             
*@$$A$@#?-_

EDIT:

Would this do the trick? Escape every unusual character with a backslash, and omit single or double quotes. (Example is in Perl but any language can do this)

sub escape {
    $_[0] =~ s/([^a-zA-Z0-9_])/\\$1/g;
    return $_[0];
}

A quite good answer is here: http://unix.stackexchange.com/q/4770/5779 — Paŭlo Ebermann, Jun 10 '11 at 15:10
The provided perl script (in the edit) is incorrect. It fails for newlines, which when escaped with a backslash are simply ignored. — Score_Under, Jan 15 '16 at 19:04

score 31 · Answer 1 · edited Dec 19 '20 at 14:02

31

If you want to securely quote anything for Bash, you can use its built-in printf %q formatting:

cat strings.txt:

yes
no
Hello, world
C:\Program Files\
"
\
\\
\\\
\\\\
\\\\\
"\
"\T
"\\T
!1
!A
"!\/'"
"Jeff's!"
$PATH
%PATH%
&
<>|&^
*@$$A$@#?-_

cat quote.sh:

#!/bin/bash
while IFS= read -r string
do
    printf '%q\n' "$string"
done < strings.txt

./quote.sh:

yes
no
Hello\,\ world
C:\\Program\ Files\\
\"
\\
\\\\
\\\\\\
\\\\\\\\
\\\\\\\\\\
\"\\
\"\\T
\"\\\\T
\!1
\!A
\"\!\\/\'\"
\"Jeff\'s\!\"
\$PATH
%PATH%
\&
\<\>\|\&\^
\*@\$\$A\$@#\?-_

These strings can be copied verbatim to for example echo to output the original strings in strings.txt.

edited Dec 19 '20 at 14:02

oguz ismail

1
16
47
69

answered Jun 10 '11 at 13:01

l0b0

55,365
30
138
223

I was expecting each of `\"!'$<>|&^` to need escaping, but I notice that printf has also escaped the `*` and the `?` in the final string. Have I missed any? Is there a complete list of characters which must be backslash-escaped? – qntm Jun 10 '11 at 13:12
1

Well, it looks like any character can be backslash-escaped without penalty, so the safest course of action is to escape everything other than letters, numbers and the underscore. In Perl, the function I'm describing is: `sub escape { $_[0] =~ s/([^a-zA-Z0-9_])/\\$1/g; return $_[0]; }` – qntm Jun 10 '11 at 18:14
@qntm: But what happens if the string (or part of it) is already escaped? You'd get double escaping, one of the five horsemen of the codepocalypse. – l0b0 Jun 28 '11 at 20:52
2

If the string is already single-escaped, then it will be printed at the command line double-escaped, and then become available inside the program single-escaped. In other words the string available in the program will be exactly the original (single-escaped) string, which is precisely what I want. – qntm Jul 01 '11 at 21:47
@TrevorHickey No, `%q` is a Bash `printf`ism. – l0b0 Mar 01 '15 at 08:10

score 28 · Answer 2 · answered Nov 27 '15 at 02:04

28

What is the general procedure for safely escaping an arbitrary string for use as a command line argument in Bash?

Replace every occurrence of ' with '\'', then put ' at the beginning and end.

Every character except for a single quote can be used verbatim in a single-quote-delimited string. There's no way to put a single quote inside a single-quote-delimited string, but that's easy enough to work around: end the string ('), then add a single quote by using a backslash to escape it (\'), then begin a new string (').

As far as I know, this will always work, with no exceptions.

answered Nov 27 '15 at 02:04

Tanner Swett

3,241
1
26
32

1

Aside from the `printf` answer (which only works if you're in bash already), this is the only correct answer in the thread. – Score_Under Jan 15 '16 at 19:03
Great answer! Easily applicable in all languages! You don't have to escape even new lines, works with everything :)! – Wojciech Kulik Aug 25 '20 at 17:59
This should be the accepted answer and deserve more upvotes – Sumit Trehan Dec 14 '20 at 19:16

score 1 · Answer 3 · answered Jun 18 '13 at 02:15

Whenever you see you don't get the desired output, use the following method:

"""\special character"""

where special character may include ! " * ^ % $ # @ ....

For instance, if you want to create a bash generating another bash file in which there is a string and you want to assign a value to that, you can have the following sample scenario:

Area="(1250,600),(1400,750)"
printf "SubArea="""\""""${Area}"""\""""\n" > test.sh
printf "echo """\$"""{SubArea}" >> test.sh

Then test.sh file will have the following code:

SubArea="(1250,600),(1400,750)"
echo ${SubArea}

As a reminder to have newline \n, we should use printf.

score 1 · Answer 4 · edited Jun 10 '11 at 13:35

1

You can use single quotes to escape strings for Bash. Note however this does not expand variables within quotes as double quotes do. In your example, the following should work:

script.pl '!foo'

From Perl, this depends on the function you are using to spawn the external process. For example, if you use the system function, you can pass arguments as parameters so there"s no need to escape them. Of course you"d still need to escape quotes for Perl:

system("/usr/bin/rm", "-fr", "/tmp/CGI_test", "/var/tmp/CGI");

edited Jun 10 '11 at 13:35

daxim

39,270
4
65
132

answered Jun 10 '11 at 12:38

jjmontes

24,679
4
39
51

2

Using single quotes doesn't seem to work if the argument contains a single quote of its own, e.g. `"Jeff's!"`. A single quote can't be backslash-escaped either. – qntm Jun 10 '11 at 13:00
@qntm: Use the following: `"Jeff's"'!'`. Think of it as two separate tokens and sits next to each other: `"Jeff's"` and `'!'` – Hai Vu Jun 10 '11 at 14:00
1

To use a single quote in a single quoted string replace it with '\'' . For example, 'Jeff'\''s!' . This is proposterous for a human, but is nice and simple for a script: first use a regexp to convert ' to '\'' and then enclose in '...'. – Dan Sheppard Jul 23 '15 at 11:17

score 1 · Answer 5 · answered Jun 10 '11 at 14:58

1

sub text_to_shell_lit(_) {
   return $_[0] if $_[0] =~ /^[a-zA-Z0-9_\-]+\z/;
   my $s = $_[0];
   $s =~ s/'/'\\''/g;
   return "'$s'";
}

See this earlier post for an example.

answered Jun 10 '11 at 14:58

ikegami

367,544
15
269
518

score 0 · Answer 6 · answered Jun 10 '11 at 12:35

0

Bash interprets exclamation marks only in interactive mode.

You can prevent this by doing:

set +o histexpand

Inside double quotes you must escape dollar signs, double quotes, backslashes and I would say that's all.

answered Jun 10 '11 at 12:35

Benoit

76,634
23
210
236

This is useful, but it would be good if I could avoid having to turn histexpand off and on for every command call. – qntm Jun 10 '11 at 13:17
if you are writing a shell script you don't have to. exclamation marks are only interpreted in interactive mode. You can turn it off in your bashrc. – Benoit Jun 10 '11 at 13:27
I'm not writing a shell script. – qntm Jun 10 '11 at 13:42

score 0 · Answer 7 · answered Jun 10 '11 at 14:10

0

This is not a complete answer, but I find it useful sometimes to combine two types of quote for a single string by concatenating them, for example echo "$HOME"'/foo!?.*' .

answered Jun 10 '11 at 14:10

micans

1,106
7
16

score 0 · Answer 8 · answered Nov 09 '20 at 03:49

0

FWIW, I wrote this function that invokes a set of arguments using different credentials. The su command required serializing all the arguments, which required escaping them all, which I did with the printf idiom suggested above.

$ escape_args_then_call_as myname whoami

escape_args_then_call_as() {
    local user=$1
    shift

    local -a args
    for i in "$@"; do
        args+=( $(printf %q "${i}") )
    done

    sudo su "${user}" -c "${args[*]}"
}

answered Nov 09 '20 at 03:49

Christopher King

1,034
1
8
21

Thanks for posting a solution on a challenge that is still relevant a decade later. But your solution removes single quotes around any of the words in the loop. Try this in shell file test (code unformatted in comments, use Edit on this post to see formatted) : ``` #!/bin/bash for i in "$@"; do args+=( $(printf %q "${i}") ) done echo "${args[*]}" ``` Then command: **./test sed -i 's/foo/bar' fname** Result: **sed -i s/foo/bar fname** – TonyG Nov 09 '20 at 21:49

How can I escape an arbitrary string for use as a command line argument in Bash?

8 Answers8

Linked

Related