What is the most pythonic way to structure a multi-module python program?

Question

I've built a fairly sizeable program in a single file. The size of the file was making it unworkable, so I decided to split into multiple modules, but have since had a major headache of variable scope. I've largely fixed it (with a fair bit of learning along the way), but I'm keen to understand good structuring to avoid future lessons learnt the hard way. There are a couple of specific points, but general advice is also welcome.

Modules that need to share the same namespace

I have two modules that seem to need to share the same namespace. One is the main flow of the program (which transfers data to and from objects, and calls the UI), the other is the UI (which responds to user input, calling the main flow).

Should each of these modules import the other, and then the main file import both? That doesn't seem particularly elegant to me.

from [modulename] import *

In the answers to this question:

Python: Sharing global variables between modules and classes therein

There is a suggestion that from [modulename] import * should be avoided.

Is it OK to use from [modulename] import * to build together a load of modules that just have class definitions? What are "safe" use cases?

Why do you think your two modules need to share the same namespace? — BrenBarn, Jun 02 '13 at 18:28
@BrenBarn so they can call each other. The program flow function does some work on the data, which dictates what needs to be presented to the user, so it calls the UI. Then the UI takes response from the user, and needs to call the program flow function to do some analysis/decision making. — user1379351, Jun 02 '13 at 18:41
Circular dependencies don't work in python. When module_a imports module_b, python hasn't gotten around to interpreting module_a past the "import module_b" statement. This means that you'll get name errors for anything that comes after the import in module_a. It sounds like you should either stick with your one big file, or spend some time redesigning, so you can factor out modules that don't depend on anything. — Seth, Jun 02 '13 at 18:42

morningstar · Accepted Answer · 2013-06-03T00:56:26.467

Modules that need to access each other's namespace is not the same as modules that need to share the same namespace. I can't think of anything you can do with from modulename import * that you can't do with import modulename. You just have to preface a lot of your names with modulename. That's a good thing, not a bad thing. It makes your code self-documenting, which is why from modulename import * is to be avoided.

You can have the UI and main flow modules import each other. The only way you'll run into problems is if you reference names between them outside of the scope of functions. For example

# mainflow.py
import ui # interpreter stops reading mainflow and starts reading ui

class Foo:
    ...

theUI = ui.UI()

# ui.py
import mainflow # mainflow already being loaded; interpretation of ui continues uninterrupted

def dosomething():
    myfoo = mainflow.Foo() # so far so good, not interpreted until the function is called

class Bar(mainflow.Foo): # mainflow.Foo not reached yet, error here
    ...

class UI:
    ...

On the other hand if ui happens to get imported first, then you get the error at theUI = ui.UI(), when all of mainflow has been interpreted but ui has only been interpreted as far as import mainflow. As long as you put all the references to each other inside functions, though, you can get along fine. E.g.

# mainflow.py
import ui
...

theUI = None

def initialize():
    global theUI
    theUI = ui.UI()

There's still a problem with the dependency between the classes; I recommend you don't do anything like that. But if you did, you could make the whole thing work with this strange approach:

# mainflow.py
...

theUI = None

def initialize():
    global theUI
    theUI = ui.UI()

import ui # Waht!? Crazy! Import at the bottom of a file. Now all of mainflow's names are guaranteed to exist and ui can access them.

Now with the first version of ui.py and the last version of mainflow.py, the program would compile and run. I don't really recommend the above; better organize your code so you don't have such dependencies. But if all you have is calls back and forth between functions in modules, you don't have to resort to such tricks.

There are more object-oriented designy ways to make your UI and your program flow not directly depend on each other, but such a redesign would be more involved than just copy and paste to files and prefacing names with module. I don't think you want to go overboard with your redesign unless you have a specific reason.

While I agree with most of what you say, there's a mistake in the code of the last part. It's not necessary to have the `theUI = None` if you have the `global theUI` statement in the `initialize()` function. — martineau, Jun 02 '13 at 23:31
It might be desirable if you ever have any code that could run before `initialize`. Then it can check if the UI has been initialized yet. — morningstar, Jun 03 '13 at 00:54

What is the most pythonic way to structure a multi-module python program?

1 Answers1