How does argparse (and the deprecated optparse) respond to 'tab' keypress after python program name, in bash?

Question

I have tested optcomplete working with the optparse module. Its example is a simple file so I could get that working. I also tested it using the argparse module as the prior one is deprecated. But I really do not understand how and by whom the python program gets called on tab presses. I suspect bash together with the shebang line and the argparse (or optparse) module are involved in some way. I have been trying to figure this out (now gonna read the source code).

I have a little more complex program structure, which includes a wrapper around the piece of code which handles the arguments. Its argparse.ArgumentParser() instantiation and calls to add_argument() - which are superclassed into another intermediate module to avoid duplicating code, and wrapper around that is being called - are inside a function.

I want to understand the way this tab completion works between bash and python (or for that matter any other interpretor like perl).

NOTE: I have a fair understanding of bash completion (which I learned just now), and I think I understand the bash(only) custom completion.

NOTE: I have read other similar SO questions, and none really answer this Q.

Edit: Here is the bash function.
I already understood how the python module gets to know about words typed in the command line, by reading os.environ values of variables

$COMP_WORDS
$COMP_CWORD
$COMP_LINE
$COMP_POINT
$COMPREPLY

These variables have values only on tab press. My question is how does the python module gets triggered?

The `optcomplet` docs say: "You also need to source a Bash function and then to tell Bash to trigger optcomplete completion for the specific programs that use it:". If you did do that, why are you still asking how it works? It should be obvious the you just need to tell bash what program to call if completion is requested. — Niklas B., Mar 05 '12 at 15:17
Sorry, I misunderstood what I read. I'll add an answer for that. — Niklas B., Mar 05 '12 at 17:30

Niklas B. · Accepted Answer · 2012-03-05T18:24:31.747

To understand what's happening here, let's check what that bash function actually does:

COMPREPLY=( $( \
    COMP_LINE=$COMP_LINE  COMP_POINT=$COMP_POINT \
    COMP_WORDS="${COMP_WORDS[*]}"  COMP_CWORD=$COMP_CWORD \
    OPTPARSE_AUTO_COMPLETE=1 $1 ) )

See the $1 at the end? That means that it actually calls the Python file we want to execute with special environment variables set! To trace what's happening, let's prepare a little script to intercept what optcomplete.autocomplete does:

#!/usr/bin/env python2
import os, sys
import optparse, optcomplete
from cStringIO import StringIO

if __name__ == '__main__':    
    parser = optparse.OptionParser()

    parser.add_option('-s', '--simple', action='store_true',
                      help="Simple really simple option without argument.")

    parser.add_option('-o', '--output', action='store',
                      help="Option that requires an argument.")

    opt = parser.add_option('-p', '--script', action='store',
                            help="Option that takes python scripts args only.")
    opt.completer = optcomplete.RegexCompleter('.*\.py')

    # debug env variables
    sys.stderr.write("\ncalled with args: %s\n" % repr(sys.argv))
    for k, v in sorted(os.environ.iteritems()):
        sys.stderr.write("  %s: %s\n" % (k, v))

    # setup capturing the actions of `optcomplete.autocomplete`
    def fake_exit(i):
      sys.stderr.write("autocomplete tried to exit with status %d\n" % i)
    sys.stdout = StringIO()
    sys.exit = fake_exit

    # Support completion for the command-line of this script.
    optcomplete.autocomplete(parser, ['.*\.tar.*'])

    sys.stderr.write("autocomplete tried to write to STDOUT:\n")
    sys.stderr.write(sys.stdout.getvalue())
    sys.stderr.write("\n")

    opts, args = parser.parse_args()

This gives us the following when we try to autocomplete it:

$ ./test.py [tab]
called with args: ['./test.py']
  ...
  COMP_CWORD: 1
  COMP_LINE: ./test.py 
  COMP_POINT: 10
  COMP_WORDS: ./test.py 
  ...
  OPTPARSE_AUTO_COMPLETE: 1
  ...
autocomplete tried to exit with status 1
autocomplete tried to write to STDOUT:
-o -h -s -p --script --simple --help --output

So optcomplete.autocomplete just reads the environment, prepares the matches, writes them to STDOUT and exits. The result -o -h -s -p --script --simple --help --output is then put into a bash array (COMPREPLY=( ... )) and returned to bash to present the choices to the user. No magic involved :)

Hey thanks a lot, in fact I noticed that $1 when I read your comment and went on the page to get the link. Then I got struck by what I already knew but never used as I never wrote any bash script - $1 is first arg.- :S. It was silly of me that I didn't get that previously. — 0xc0de, Mar 05 '12 at 18:14

How does argparse (and the deprecated optparse) respond to 'tab' keypress after python program name, in bash?

1 Answers1

Linked