2

I am completely stuck at mixing Python with Bash and it is really tough on me.
I know that using double vs single quotes in Bash depends on your intention to use variables, escapes and special characters.
I also know, that using double vs single quotes in Python is matter of taste mostly and PEP8 also confirms this.

But what if I want to use Python snippet inside Bash like this? It works either way.

#!/bin/bash
arg3="something special"
python << END
import functions; 
print functions.do_smth('$arg', '$arg2')    
print len("$arg3");
END

Does Python or Bash rules apply for the quotes in the above snippet?

Is it different from such allocation of Python code?

python -c "import functions; print functions.do_smth('$arg', '$arg2'); print len("$arg3");"

Does it matter for quoting when I use 1st or 2nd variant? Does it matter if I pass variable to function or simply use it?

python -c "print 'great program eh'"
python -c "import special; do_smth_special_with('great program eh')"

A have completely no idea of the differences between all these use-cases!
Especially many problems I face when I should pass JSON output as variable (which also has goddamn quotes and escapes!) from Bash to Python function.

Really need thorough explanation here.

Community
  • 1
  • 1
Suncatcher
  • 10,355
  • 10
  • 52
  • 90
  • Avoid mixing code like that. Similar things like what can happen with SQL injections can happen here, too. – blubberdiblub Apr 29 '17 at 19:07
  • A `heredoc` is it's own quoting mechanism, quotes and single-quotes in the `heredoc` are not interpreted by `bash`. So in all cases `$arg3` will be expanded unless you escape the `\$`. The `python -c "..."` means bash treats everything as double quoted and will expand `$` even in single quotes. But you will need to escape `\"` inside the string or bash will terminate it. – AChampion Apr 29 '17 at 19:16
  • The other case you are missing is the `` marks, which are left of the #1 or ~, that will execute a shell process inside of a quotation – oliver Apr 29 '17 at 19:37
  • @Oliver, could you point me to the docu about ``? – Suncatcher Apr 30 '17 at 06:53
  • @blubberdiblub, you mean to avoid using heredocs? Or what mixing do you mean? – Suncatcher Apr 30 '17 at 06:55
  • @Suncatcher no, not heredocs in general. They are useful. But the way heredocs are used in the example above: To execute them as Python code and interpolate variables into it. That's not only atrocious, but also dangerous. The least that could happen is having one of the matching kind of quote in the variable and Python bails right at the start with a syntax error. But just think of all the code that could be injected unintentionally. – blubberdiblub Apr 30 '17 at 07:01
  • @Suncatcher on a more constructive note: Please pass variables as command line arguments instead of interpolating them into the source code. – blubberdiblub Apr 30 '17 at 07:17
  • @blubberdiblub, give the example of such passing. You say heredocs are useful but on the next line you say to use `-c` arguments. Your statements contradict to each other. – Suncatcher Apr 30 '17 at 07:30
  • @Suncatcher no, they don't. I'm talking about normal command line arguments which you can also pass to normally run Python scripts. What I'm talking about is to **not interpolate into source code**. For a standard script, passing command line arguments would look like `python script.py "$arg" "$arg2" "$arg3"` (Note the double quotes to make it safe). For your `-c` version this would be `python3 -c 'import sys; print("arg={!r} arg2={!r} arg3={!r}".format(*sys.argv[1:]))' "$arg" "$arg2" "$arg3"`. Analogously for heredocs. – blubberdiblub Apr 30 '17 at 07:43
  • Okay, got it. I know this way, but I don't want to spawn miriads of py scripts in my program. Heredocs seem more handy to me. – Suncatcher Apr 30 '17 at 07:48
  • It works for heredocs too. You can pass command line arguments there as well. Just don't put the bash variables directly into the middle of the source code (= interpolation). You understand that that would be dangerous, right? You know how SQL injections work and that putting unescaped variables in the middle of executed source code is similar? – blubberdiblub Apr 30 '17 at 07:52
  • @Suncatcher here is some information from the Beginners Guidebook: http://www.tldp.org/LDP/Bash-Beginners-Guide/html/Bash-Beginners-Guide.html#sect_03_03 – oliver May 01 '17 at 00:34

1 Answers1

1

Does Python or Bash rules apply for the quotes in [an unquoted heredoc]?

Python rules apply to the quotes; bash rules apply to the $ sigils. Since bash never looks at the quotes, it doesn't matter to bash what type of quotes they are; the $foo gets replaced regardless (unless you use \$; bash looks at backslashes).

If you'd quoted the heredoc delimiter (<<<"END"), then bash would not attempt to replace variable expansions nor interpret backslashes.

The rules for heredocs are explained thoroughly in the bash manual.

Is it different from [python -c "…"]?

Yes, bash interprets command-line arguments before passing them to the program, so bash rules apply and python only sees whatever quotes which were passed through.

The rules for words in the command line are explained thoroughly in the bash manual (read this section in context).

rici
  • 234,347
  • 28
  • 237
  • 341
  • Thanks for pointing to heredoc help. So Python rules apply to heredocs and Bash rules apply to `-c` argument and Python only sees the string left after unfolding the quotes. If I write `python -c "print '$var'"` then Python will receive `$var` string, and if `python -c "print "$var""` then Python will receive $var value. Correct? – Suncatcher Apr 30 '17 at 06:49
  • So the only way to suppress expansion is escaping quotes? – Suncatcher Apr 30 '17 at 08:37
  • @suncatcher: in both those examples, bash expands `$var`. In `python -c "print '$var'"`, `$var` is inside double quotes; bash quotes don't nest. In `python -c "print "$var""` `$var` is unquoted, becausee bash matches the quotes in pairs: first " print ", then the unquoted value of `$var` (word-split and glob expanded), then `"" ` which is empty. – rici Apr 30 '17 at 08:43
  • So in this example we have actually **three** quote pairs? And what quote syntax I should use to suppress expansion of $var? I often have problems with passing JSONs into Python. They have `'` and `\ ` in their body and Bash often splits my JSON into thousands of arguments treating inner quotes as delimiters. – Suncatcher Apr 30 '17 at 08:52
  • @suncatcher: `python -c 'print "$var"'`. If you want to see what bash hands to python, replace `python -c` with `echo`. – rici Apr 30 '17 at 08:54
  • `"print "$var"" ` and `'print '$var''` are identical. So is `"print "$var''`. – rici Apr 30 '17 at 08:58
  • Something bizarre for me:) I've never met such strange nesting behavior of quotes, i.e. sequential instead of hierarchical. Could you give link to documentation where I can read about it? – Suncatcher Apr 30 '17 at 09:06