4 apostrophes in python code

Question

Reference https://github.com/noxrepo/pox/blob/carp/pox.py

I'm trying to understand what does the 4 apostrophes means? It doesn't look like commenting and near the end of the code there were another 3 apostrophes. Can someone help to explain the code below?

#!/bin/sh -

''''true
#export OPT="-u -O"
export OPT="-u"
export FLG=""
if [ "$(basename $0)" = "debug-pox.py" ]; then
  export OPT=""
  export FLG="--debug"
fi

if [ -x pypy/bin/pypy ]; then
  exec pypy/bin/pypy $OPT "$0" $FLG "$@"
fi

if type python2.7 > /dev/null 2> /dev/null; then
  exec python2.7 $OPT "$0" $FLG "$@"
fi
exec python $OPT "$0" $FLG "$@"
'''
from pox.boot import boot
if __name__ == '__main__':
  boot()

any additional apostrophes are considered apart of the block — 0TTT0, Nov 07 '17 at 02:33
It is just a typo. 3 single quotes (or 3 double quotes) will create a quote block until the next set of 3 single quotes is read. — James, Nov 07 '17 at 02:46
@cricket_007 The script is both valid Python and valid shell. As Python code, it's a docstring that contains a shell script. As shell, `''''true` evaluates to the same string as `true` after quote removal. The next `'''` will technically produce an error, but the preceding `exec` prevents the shell from ever seeing the rest of the script. — chepner, Nov 07 '17 at 14:04
python and shell script can work together in such manner? Is this legit? It's really frustrating not able to find this item in python manual... — user3782604, Nov 07 '17 at 14:18
It's a bit of a hack that has nothing to do with Python itself. Someone thought it would be clever to do this instead of writing a separate shell script to set up the environment and execute a second "pure" Python script. I've added an answer to try to explain in greater detail how it works in both languages. — chepner, Nov 07 '17 at 14:20
https://en.wikipedia.org/wiki/Polyglot_(computing) will make for interesting reading. — chepner, Nov 07 '17 at 14:41

Raymond Hettinger · Accepted Answer · 2017-11-07T04:31:43.060

Overall Answer

The first three apostrophes start a multi-line string. The next apostrophe is just part of the contents of the string.

Inspecting the Result

The script stores the string in the __doc__ variable. After running the code interactively with python -i pox.py, it is easy to see the parsed docstring directly:

>>> print __doc__
'true
#export OPT="-u -O"
export OPT="-u"
export FLG=""
if [ "$(basename $0)" = "debug-pox.py" ]; then
  export OPT=""
  export FLG="--debug"
fi

if [ -x pypy/bin/pypy ]; then
  exec pypy/bin/pypy $OPT "$0" $FLG "$@"
fi

if type python2.7 > /dev/null 2> /dev/null; then
  exec python2.7 $OPT "$0" $FLG "$@"
fi
exec python $OPT "$0" $FLG "$@"

Note how the fourth apostrophe was kept as part of the docstring.

Details

According to the tokenize module, here is how Python views the above code:

NL        : '\n'
COMMENT   : '#!/bin/sh -'
NL        : '\n'
NL        : '\n'
STRING    : '\'\'\'\'true\n#export OPT="-u -O"\nexport OPT="-u"\nexport FLG=""\nif [ "$(basename $0)" = "debug-pox.py" ]; then\n  export OPT=""\n  export FLG="--debug"\nfi\n\nif [ -x pypy/bin/pypy ]; then\n  exec pypy/bin/pypy $OPT "$0" $FLG "$@"\nfi\n\nif type python2.7 > /dev/null 2> /dev/null; then\n  exec python2.7 $OPT "$0" $FLG "$@"\nfi\nexec python $OPT "$0" $FLG "$@"\n\'\'\''
NEWLINE   : '\n'
NAME      : 'from'
NAME      : 'pox'
OP        : '.'
NAME      : 'boot'
NAME      : 'import'
NAME      : 'boot'
NEWLINE   : '\n'
NAME      : 'if'
NAME      : '__name__'
OP        : '=='
STRING    : "'__main__'"
OP        : ':'
NEWLINE   : '\n'
INDENT    : '  '
NAME      : 'boot'
OP        : '('
OP        : ')'
NEWLINE   : '\n'
DEDENT    : ''
ENDMARKER : ''

Tokenization Script

Here is a Python 2.7 script that tokenizes the pox.py script:

from __future__ import print_function
import tokenize
import token

with open('pox.py') as f:
    for tok in tokenize.generate_tokens(f.readline):
        tok_type, tok_str, (srow, scol), (erow, ecol), logical_lineno = tok
        print('%-10s: %r' % (token.tok_name[tok_type], tok_str))

chepner · Answer 2 · 2017-11-07T14:37:55.650

This is a file that can be executed as both a shell script and a Python script. Note the shebang is #!/bin/sh, so the script is intended to be run as a shell script first. (I'm ignoring the - argument, which would indicate that sh should read from standard input instead of a file. I'm not quite sure what it's significance is here, as I've never seen a shebang written that way.)

The intended invocation is something like ./pox.py ... or sh ./pox.py ...

The first line of the script is ''''true, which after quote removal is the same as true, so that command runs and does nothing except exit with success. The following lines are valid commands to determine which Python interpreter to use and what options to pass to the invocation. One of the three exec commands

exec pypy/bin/pypy $OPT "$0" $FLG "$@"
exec python2.7 $OPT "$0" $FLG "$@"
exec python $OPT "$0" $FLG "$@"

will then execute the same file as a Python script, so the lines beginning with the second ''' are never seen by the shell (which is good, because the word started by the first ' is neither terminated with a final closing ', nor would the string be a valid command if it were terminated). "$0" is the name of the current file, and "$@" represents the arguments to the script (which are ignored by the shell script except to pass on to the Python script, as shown here).

Once the script is run as a Python script, the line starting ''''true is simply seen as the beginning of a doc string that will be ignored. The Python script proper is simply

from pox.boot import boot
if __name__ == '__main__':
  boot()

As to why this all has to start with ''''true? As a shell script, you need to have an even number of quotes to balance each other. However, '''' alone would be an empty string that the shell would attempt to treat as a command, and there is no command whose name is the empty string. ''''true, however, does result in a valid command.

4 apostrophes in python code

2 Answers2

Overall Answer

Inspecting the Result

Details

Tokenization Script