python Sphinx "the module executes module level statement and it might call sys.exit()."

Question

I try to use Sphinx for documenting my code. But I see the error:

the module executes module level statement and it might call sys.exit().

I found that this error is associated with the code:

import argparse
# parse command line arguments
parser = argparse.ArgumentParser(description='AWS VPN status checker.')
parser.add_argument('account', type=str, help='AWS account name.')
parser.add_argument('region', type=str, help='region of VPN tunnel.')
parser.add_argument('ipaddress', type=str, help='Tunnel IP address.')
parser.add_argument("-d", "--debug", help="debug", action="store_true")
args = parser.parse_args()

I think it is related to "side effects" when I import module.

Why is it bad and how can I fix this?

*"Why is it bad"* - because Sphinx imports your code to analyse it, and if there's code running at the top level of the module it could, per the message, exit before Sphinx has finished. *"how can I fix this?"* - **don't** have code running at the top level. Move it into a function and add `if __name__ == '__main__'` to call it (see http://stackoverflow.com/q/419163/3001761). — jonrsharpe, Jan 02 '16 at 20:12

score 8 · Answer 1 · edited Jun 20 '20 at 09:12

The Sphinx Warning is an effect, caused by a preceding parse_args() error. The arg_parse() function is called, finds a normal error in the arguments it's expected to parse, and exists.

Invalid arguments

While parsing the command line, parse_args() checks for a variety of errors, including ambiguous options, invalid types, invalid options, wrong number of positional arguments, etc. When it encounters such an error, it exits and prints the error along with a usage message:

The most likely reason parse_args() exits while generating Sphinx documentation, is because what you are really calling is sphinx-build or make html. So what is being executed on your python shell is the following signature:

sphinx-build

Synopsis

sphinx-build [options] < sourcedir > < outputdir > [filenames …]

Meaning you probably aren't executing your script with the arguments you coded ArgumentParser to require. Either run sphinx-build or make html including the command line arguments arg_parse() requires inArgumentParser(), or don't call arg_parse() when generating documentation.

How to fix this?

One possible approach is the following:

entry_script.py:

from sys import argv
from pathlib import Path

import cmd_line_module

# Checking what the command line contains can be useful.
print(argv)

EXAMPLE_ARGS = ['-i', '../in_dir_test', '-o', 'out_dir_test']

# Script.
if __name__ == "__main__":

    # default Namespace
    print(cmd_line_params.args)

    # command-line
    cmd_line_module.set_args()
    print(cmd_line_params.args)

    # test case
    cmd_line_module.set_args(EXAMPLE_ARGS)
    print(cmd_line_params.args)

# Sphinx-build or make.
elif Path(argv[0]).name == "sphinx-build" or Path(argv[0]).name == "build.py":

    cmd_line_module.set_args(EXAMPLE_ARGS)

# Module only.
else:
    cmd_line_module.set_args(EXAMPLE_ARGS)

cmd_line_module.py:

import argparse


_DEFAULT = argparse.Namespace(in_dir=None, out_dir=None)
args = _DEFAULT


def command_line_args():

    parser = argparse.ArgumentParser(prog='entry_script', description='Does magic :) .')
    parser.add_argument("-i", "--in_dir", help="Input directory/file. Use absolute or relative path.")
    parser.add_argument("-o", "--out_dir", help="Output directory. Use absolute or relative path.")

    return parser


def set_args(cmd_line=None):

    parser = command_line_args()
    global args
    args = parser.parse_args(cmd_line)

A few notes on the solution might prove useful to the readers:

1. The cmd_line_module.py maintains args as a variable on the module level, to be as similar to the argparse tutorial example as possible. Specific Sphinx extensions for argparse can be found on this thread.

2. Using a default Namespace for args might be convenient, it's included as a suggestion. (Expected default values can help tests in importing modules).

3. Testing for __main __ or sphinx-build may not be necessary depending, the 3 if tests are included only to add context to the question.

4. Use of DEFAULT_ARGS shows how to use parse_args() without reading from sys.argv, and also how you can run sphinx-build dispensing use of if __name __ == "__main __": (should you find it convenient for any reason)...

5. The name and path of the sphinx-build script may vary across operating systems.

Final note: If you like to write modules that self-initialize their variables (like me) pay special attention to run parse_args() before importing modules that might have variables depending on it.

6.1. More on Modules

A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement. (They are also run if the file is executed as a script.)

score 1 · Answer 2 · answered Nov 18 '19 at 13:33

In my case, I had a submodule with 3rd party library import and that library was a source of a problem (pyautogui). But I spent a few hours trying to figure it out because errors were like:

FOO.a import error the module executes module level statement and it might call sys.exit()
FOO.b import error the module executes module level statement and it might call sys.exit()
FOO.d.foo import error the module executes module level statement and it might call sys.exit()
FOO.d.bar import error the module executes module level statement and it might call sys.exit()
FOO.d.baz import error the module executes module level statement and it might call sys.exit()
FOO.d import error the module executes module level statement and it might call sys.exit()
FOO import error the module executes module level statement and it might call sys.exit()

While I had package structure similar to this:

FOO
├── a
├── b
│   ├── ci
|   └── __init__.py|
└── d
    ├── foo
    ├── bar
    ├── baz
    └── __init__.py

Where a and b had import d string in them. And import pyautogui was in FOO.d.bar submodule.

I got rid of 3rd party library that caused problems. – banderlog013 Feb 23 '20 at 23:46 — banderlog013, Feb 23 '20 at 23:46

score 1 · Answer 3 · answered Nov 20 '20 at 14:43

I had this issue too. My solution was to add to the script before the argparse code this if statement:

if __name__ == "__main__":
   parser = argparse.ArgumentParser(description="some description", formatter_class=RawTextHelpFormatter)
   parser.add_argument(....)
   args = parser.parse_args()
   ......

python Sphinx "the module executes module level statement and it might call sys.exit()."

3 Answers3

Linked