Python Argparse, How to Properly Organize ArgParse Code

Question

I haven't used argparse in 10 years, but I understand it and what I have below does work as I want it to work... but this is going to get a lot more complicated as I continue adding commands, parsers and subparsers. I'm wondering what the best way is to organize this? In my mind I should be able to see the command sequence in the text nearly as clearly as I see it in my diagram... but everytime I look at it, after being away for a while, my brain swims as I try to follow it. There has to be a better way to organize this right?

import argparse
from modules import cli_tools

#LVL 1: create the top-level parser for the "sacs" command.
sacs_parser = argparse.ArgumentParser(prog = 'sacs', description = 'Master Command For Controlling SACS.')
sacs_subparsers = sacs_parser.add_subparsers(help='Management Module Selector.')

#LVL 2: create the second-level parsers for the "sacs [module]" commands.
csv_parser = sacs_subparsers.add_parser('csv', help='Generic CSV Management Module.')
am_parser = sacs_subparsers.add_parser('am', help='SACS Asset Management Module.')
mm_parser = sacs_subparsers.add_parser('mm', help='SACS Metric Management Module.')

#LVL 3: create the third-level subparser for the "sacs [module] [action]" commands.
csv_action_subparser = csv_parser.add_subparsers(help='The action to perform.')
mm_action_subparser = mm_parser.add_subparsers(help='The action to perform.')

#LVL 4: create the fourth-level subparser for the "sacs [module] [action] [type]" commands.
mm_create_parser = mm_action_subparser.add_parser('create', help='Used to Create a new event/asset input file.')
mm_create_type_parser = mm_create_parser.add_subparsers(help='The type of file to create.')

#LVL 5: create the fifth-level parser for the "sacs [module] [action] [type]" commands.
csv_reconcile_parser = csv_action_subparser.add_parser('reconcile', help='reconcile two csvs.')
mm_create_asset_parser = mm_create_type_parser.add_parser('assets', help='Create an Asset File.')
mm_create_asset_subtype_parser = mm_create_asset_parser.add_subparsers(help='The type of file to create.')
mm_create_event_parser = mm_create_type_parser.add_parser('events', help='Create an Event File.')

#LVL 6: create the sixth-level parser for the "sacs [module] [action] [type] [subtype]" commands.
mm_create_asset_uaid_parser = mm_create_asset_subtype_parser.add_parser('uaid', help='Create an Asset File with UAID as the primary key.')
mm_create_asset_vid_parser = mm_create_asset_subtype_parser.add_parser('vid', help='Create an Asset File with Vulnerability ID as the primary key.')

#COMMAND ARGS: Add Arguments to the final command "sacs csv reconcile [args]"
csv_reconcile_parser.add_argument('key', help='The name of the field that holds the unique ID to compare against.')
csv_reconcile_parser.add_argument('inputfile1', help='The master file (used when same record exists in both files).')
csv_reconcile_parser.add_argument('inputfile2', help='The secondary file, which is trumped by the master file.')
csv_reconcile_parser.add_argument('outputfile', help='The output file; note it will be overwritten if it exists.')
csv_reconcile_parser.set_defaults(func=cli_tools.csv_reconcile)

#COMMAND ARGS: Add Arguments to the final command "sacs mm create assets uaid [args]"
mm_create_asset_uaid_parser.add_argument('appmapp_file', help='The input file.')
mm_create_asset_uaid_parser.add_argument('output_file', help='The output file.')
mm_create_asset_uaid_parser.set_defaults(func=cli_tools.asset_create_uaid)

#COMMAND ARGS: Add Arguments to the final command "sacs mm create assets vid [args]"
mm_create_asset_vid_parser.add_argument('vulnerability_file', help='The input file.')
mm_create_asset_vid_parser.add_argument('appmapp_file', help='The output file.')
mm_create_asset_vid_parser.add_argument('output_file', help='The output file.')
mm_create_asset_vid_parser.set_defaults(func=cli_tools.asset_create_vid)

args = sacs_parser.parse_args()
args.func(args)

Potential avenues to better way:

1. parser/subparser renaming.
2. change the ordering of the statements.
3. some way to indent without messing with python.

All ideas are on the table, I want to see how others deal with this while designing complex commands.

Answer 1

The usual way of adding clarity to Python code is to package steps in functions and even classes. argparse itself is a set of classes. Each of your parsers (including subparsers) are argparse.ArgumentParser objects. Each add_argument creates an argparse.Action subclass object. The add_subparsers creates (and returns) a specialized Action subclass that handles subparsers. And finally parse_args returns an argparse.Namespace object.

Whether any of that is applicable in your case, I don't know. Your code is readable, so I easily get an idea of what you are doing. I've never seen anyone use so many levels of subparsers. In fact using more than one level is something of a novelty (judging from some earlier SO questions).

I like to stress that the first job of argparse is to figure out what your user wants. The parsing is the first priority. Secondly it should make it easy to express what they want. I wonder whether this multilevel subparsing is easy to use. The way that argparse splits the help among subparsers makes it hard to display an big overview.

At some point big packages start to write their own helps, and even customize the parser to suit their needs.

You might get a better answer on CodeReview. The regulars there seem to more interested in code organization and naming issues. SO is more oriented to problem solving. I discourage referrals to CR if the topic is too specialized, but this is more of a general Python question than an argparse specific one. But read up on what CR expects in a question.

Multiple level argparse subparsers

Argparse with required subparser

argparse subparser monolithic help output

Python argparser repeat subparse