3

I have a script which loads various modules to do its job. Some of these modules are heavily studded with assert statements - enough to potentially cause significant slow-downs in actual use. (I'm doing a lot of operations which involve mutating graphs, so there are lots of asserts which iterate over the entire graph in order to enforce invariants or check the correctness of results.) So I want to allow users to disable these checks if they wish. Seems simple enough; this is exactly what the -O flag does, right?

However, the problem is that -O is a flag handled by the Python interpreter. I don't want to force my users to explicitly call the interpreter just to access this functionality - I want to support MyScript.py -O, rather than python -O MyScript.py. Is there any way I can do this?

I know that __debug__ cannot be set at run-time; it is special-cased into the language as an immutable constant. However, none of the problematic assert statements are in the main script, and I could easily delay loading my other modules until after I've finished parsing the command-line arguments. So if there's either some way to set the -O flag from within Python or some sort of import-with-flags functionality, that would do it.

Note 1: To be clear, it is not sufficient just to disable assert statements; there are also pieces of code which are conditional on the __debug__ constant (e.g. if (__debug__): for x in large_expensive_set: assert some_property_of(x)) which needs to not be executed.

Note 2: On the other hand, I'm not concerned about code memory size, only speed, and my speed concerns are based around expensive functions in assert statements and expensive loops in debug-only code. I know that Python's code generator, when running in -O mode, omits assert statements and if (__debug__): blocks from the compiled code, but it's fine if that doesn't happen - I don't mind if the debug-only statements are present, as long as they don't execute.

Note 3: I've considered just passing around a manual debug flag and replacing all the asserts with if (self.__debug and not assert_condition): raise AssertionError(). However, that solution:

  • Is horribly less readable
  • Is much less efficient
  • Requires changing lots of code
  • Requires non-trivial refactoring (every object has to take a debug flag, store it, and pass it on to every child object it spawns)
macdjord
  • 517
  • 2
  • 4
  • 13

1 Answers1

3

You can simply relaunch yourself if __debug__ is set. At the top of your script, try this:

if __debug__:
    import os
    import sys
    os.execv(sys.executable, ['python', '-O'] + sys.argv)

This replaces the running process with a new one using python -O. Note that os.execv never returns--it really does replace the running process.

One quirk of this approach is that it does not preserve options to the Python interpreter, so if you run python -m cProfile myscript it will relaunch without cProfile. I'm not sure how to fix this...sys.flags tells you which boolean options were set, but it doesn't tell you about -m arguments. I thought this was interesting by itself, so created a separate question about it here: Retrieve the command line arguments of the Python interpreter

Community
  • 1
  • 1
John Zwinck
  • 239,568
  • 38
  • 324
  • 436