0

I call a program in bash like: ./program $OPTION

I used an option, for instance OPTION="--executor-cores 10 --executor-memory 11", in this way it is working pretty fine,

But if the option contains quotation mark, the bash can not parse it anymore. For instance, OPTION is: --conf spark.executor.extraJavaOptions="-XX:+UseG1GC -XX:-ResizePLAB"

In this case the bash just can not recognize the option as it contains double quotations. How to solve this problem?

Rui
  • 3,454
  • 6
  • 37
  • 70

1 Answers1

1

You can't use a scalar variable (a single string) to store a list of strings. That is to say, the following won't reliably work for arbitrary data, for reasons explained at length in BashFAQ #50, and touched on below[1]:

OPTION="--executor-cores 10 --executor-memory 11"
yourprogram $OPTION

Instead, use an array -- a data type natively built to store a list of strings:

options=( --executor-cores 10 --executor-memory 11 )
yourprogram "${options[@]}"

...which will work correctly even with complex options:

options=(
  --executor-cores 10
  --executor-memory 11
  --conf spark.executor.extraJavaOptions="-XX:+UseG1GC -XX:-ResizePLAB"
)
yourprogram "${options[@]}"

[1] - Bash parsing happens in phases, and the phase where quotes are parsed as syntax happens before the parts where variable expansions take place -- so after parameter expansion (replacing $OPTION with its contents) has happened, it's too late in the process for quotes to be honored. (That's true unless you use eval to go back to the very beginning of the parsing process -- but doing that introduces a host of bugs, security-related and otherwise; see also BashFAQ #48).

This is a feature, not a bug: Were it otherwise, writing secure code in bash handling data from untrusted sources would be nearly impossible.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441