Python subprocess with apostrophes, removes them

Question

I'm running a command to my containerized Hbase DB from the subprocess.run function:

subprocess.run(('docker exec hbase bash -c "echo -e create "myTable", "R" | hbase shell"'), shell=True)

It seems that the command runs as I expected, it opens the HBase shell and then trying to execute the next command but without apostrophes: create myTable, R

The error I get:

2021-06-01 05:23:18,587 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.2.4, r67779d1a325a4f78a468af3339e73bf075888bac, 2020年 03月 11日 星期三 12:57:39 CST
Took 0.0032 seconds
stty: 'standard input': Inappropriate ioctl for device
create myTable, R
NameError: undefined local variable or method `myTable' for main:Object

HBase expects apostrophes around myTable and R. I tried to run the same command prefixing the apostrophes with :

subprocess.run(('docker exec hbase bash -c "echo -e create \"myTable\", \"R\" | hbase shell"'), shell=True)

But I get the same results.

Any idea how can I prevent it from deleting the apostrophes? Thanks in advance!

try switching apostrophes? like double on the outside and single on the inside? — Shrey Joshi, Jun 01 '21 at 05:43
If you're using `shell=True` you shouldn't put the command in a list. — Barmar, Jun 01 '21 at 05:44
does this work: `subprocess.run(("docker exec hbase bash -c 'echo -e create \'myTable\', \'R\' | hbase shell\""), shell=True)` — Shrey Joshi, Jun 01 '21 at 05:45

tripleee · Accepted Answer · 2021-06-01T05:56:21.180

You can't nest double quotes like that. Assuming we can correctly guess what you are attempting to say, try

subprocess.run([
    'docker', 'exec', 'hbase', 'bash', '-c',
    '''echo 'create "myTable", "R"' | hbase shell'''])

which also conveniently gets rid of the pesky shell=True. (I also took out the -e from echo -e which didn't seem to be contributing anything useful. Probably prefer printf if you actually need that feature, anyway.)

Python provides triple quotes specifically so that you have a way to pass in both single and double quotes in a string if you need to. The shell accepts either single or double quotes around a string (though they have slightly different semantics -- single quotes quote completely verbatim, while double quotes are somewhat weaker) and so we use those around the argument to echo. Not quoting the argument would cause bash to parse and remove the quotes. (In the shell itself, you could equivalently say echo "create \"myTable\", \"R\"" inside double quotes, with the literal double quotes escaped; but getting that through two additional layers of backslash quoting is going to get ugly. Both Python and bash -c want to do things with those backslashes.)

Python subprocess with apostrophes, removes them

1 Answers1