How to pass quoted args to GNU Parallel

Question

I need to pass some text that includes whitespace and other characters to a script that's being run by GNU Parallel.

Here is a very simple example:

$ seq 1 3 | parallel echo "Quoted ' (text)"

The above example will output this:

sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file

However, if I do this everything works:

seq 1 3 | parallel echo "\"Quoted ' (text)\""

I happen to be running this from a python script, so before passing the arguments I'm double quoting them in the script like this:

args = ["Some arg", "Another arg", "etc."]
args = ' '.join(pipes.quote(pipes.quote(arg)) for arg in args)

But it doesn't seem like a clean solution.

Does anybody know of a better way to pass arguments to GNU Parallel?

Thanks!

'not a clean solution', but does it work? Good luck. – shellter Nov 22 '11 at 16:46 — shellter, Nov 22 '11 at 16:46

score 22 · Accepted Answer · edited Dec 17 '18 at 01:45

22

zsh-4.3.12[sysadmin]% print -l {1..3} | 
  parallel -q echo "Quoted ' (text)"
Quoted ' (text) 1
Quoted ' (text) 2
Quoted ' (text) 3

As described by @mortehu:

Arguments passed to commands through parallel are expanded by the shell twice: once in the invocation of parallel, and once when parallel runs your command. -q prevents the second shell expansion.

edited Dec 17 '18 at 01:45

chrylis -cautiouslyoptimistic-

75,269
21
115
152

answered Nov 22 '11 at 17:26

Dimitre Radoulov

27,252
4
40
48

2

Thank you! I looked the man page for something so simple but could not find. Thank you for answering so clearly and not saying something like "Read the man page!" So simple. – chaimp Nov 22 '11 at 17:45
2

To make this a bit clearer: Arguments passed to commands through parallel are expanded by the shell twice: once in the invocation of parallel, and once when parallel runs your command. `-q` prevents the second shell expansion. – mortehu Feb 27 '18 at 15:05

score 12 · Answer 2 · edited Mar 10 '15 at 13:39

12

There is a whole section in the man page dedicated to quoting:

http://www.gnu.org/s/parallel/man.html#QUOTING

It even mentions the very error messages you write in your question.

If you can write it better please email your version to: parallel@gnu.org.

edited Mar 10 '15 at 13:39

nelaaro

3,006
5
38
56

answered Nov 23 '11 at 15:45

Ole Tange

1,990
16
10

How to pass quoted args to GNU Parallel

2 Answers2