0

I want to execute the following command by using sh -c (because I want to append similar commands later):

impala-shell -q "CREATE TABLE test LIKE PARQUET 'hdfs://.../foo.parquet' STORED AS PARQUET"

This command works fine. But if I try 

sh -c 'impala-shell -q "CREATE TABLE test LIKE PARQUET 'hdfs://../foo.parquet' STORED AS PARQUET"'

I get a syntax error:

LIKE PARQUET hdfs://.../...
             ^

There must be something wrong with the single quotes from the hdfs path but I can't figure it out. The hdfs path needs to be in quotes. I also tried to backslash them with /' which should actually work according to the docs. I hope someone can help me with that.

AndroidStorm
  • 161
  • 4
  • 16
  • Which docs told you backslashes would work in single-quotes? They're literal when quoted that way in all POSIX-compliant shells. – Charles Duffy Aug 15 '18 at 16:06

1 Answers1

2

You can use Python to tell you how to quote a shell command. Yes, really. :)

python -c '
import sys
try:
    from pipes import quote
except ImportError:
    from shlex import quote
print(quote(sys.stdin.read().rstrip("\n")))
' <<'EOF'
impala-shell -q "CREATE TABLE test LIKE PARQUET 'hdfs://.../foo.parquet' STORED AS PARQUET"
EOF

...emits as output:

'impala-shell -q "CREATE TABLE test LIKE PARQUET '"'"'hdfs://.../foo.parquet'"'"' STORED AS PARQUET"'

...and indeed, you can succesfully run:

sh -c 'impala-shell -q "CREATE TABLE test LIKE PARQUET '"'"'hdfs://.../foo.parquet'"'"' STORED AS PARQUET"'

Because everything in single-quotes is literal, including backslashes, you need to change to a different quoting context in order to include a single quote literal in a single-quoted string.

That's what '"'"' does: First, it ends the single-quoted context; then it enters a double-quoted context; then it inserts a literal '; then it ends the double-quoted context; then it goes back into a single-quoted context.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441