TL;DR version:
How do I have multiple submodules in a project which are able to see each other? Pythonically?
Long version:
I'm looking for a good/proper way to structure some Python code. (I'm mostly using 2.7, but --if it's different-- I'm also interested in responses for 3, which I also use.)
Basically, it involves submodules which should be able to see each other... but I'll explain in more detail below.
I'm probably just not using the right keywords to search, but everything I've found online has simply been discussing importing along a single "branch" of submodules, whereas I want to be able to import "branch-to-branch." (With one semi-exception, referenced below.)
Perhaps this is not Pythonic, I don't know. But then I'd like to know what IS the proper Python way of handling this.
Without going into too much detail about my actual problem application for proprietary reasons, I'll describe an almost exactly analogous problem:
Racecars!
Say I'm modeling racecars and the various parts of which they consist, such as tires.
Each part is modeled as an object, with various attributes and a standard API. This makes it easy to come up with a new tire class, and --so long as it fits the API-- I can just plug and play.
The same goes for other components like engines or windshields.
Then of course there are a bunch of standard utility functions which most of these things use -- maybe custom math functions or file IO functions or the like.
(Maybe I have a standard file format for engine data, and some engine types read in one or two of those files in order to give results. And maybe further still, that same format is used for other data too, such as tire stiffness based on temperature and humidity, so tires need to be able to use that file IO function too.)
Then I have different sorts of simulation objects which get populated with instances of racecars and their components, and perform certain calculations to find out different things.
Then I also have plotter objects which take simulation objects and create plots of various related things.
I could very easily put all of this in a single huge file, with something like the following sections and pseudo-code:
# (1) Utility functions
# (2) Material_Rubber (abstract) class and (non-abstract) child classes
# (3) Tire (abstract) class and (non-abstract) child classes
# (4) Windshield (abstract) class and (non-abstract) child classes
# ... AND SO ON ...
# (5) Racecar (abstract) class and (non-abstract) child classes
# (6) Track (abstract) class and (non-abstract) child classes
# (7) Weather (abstract) class and (non-abstract) child classes
# ... YOU GET THE IDEA ...
# (8) Simulation (abstract) class and (non-abstract) child classes
# (9) Plotter (abstract) class and (non-abstract) child classes
# (10) Instantiation of Material_Rubber,
# Tire,
# Windshield,
# Racecar,
# Track, and
# Weather child classes which are appropriate for
# the particular simulation I want to run, some of
# them needing each other -- eg:
hard_rubber = Material_Rubber_Hard(42)
fast_tires = Tire_Fast(hard_rubber)
smooth_windshield = Windshield_Smooth(0.42)
fast_car = Racecar_Fast(
tires = fast_tires,
windshield = smooth_windshield
)
# (11) Instantiation of Simulation child class, which is likely to be
# dependent upon most of the 'large concept level' instances above.
# Something like:
sim = Simulation_Speed(
car1 = fast_car,
car2 = medium_car,
track = normal_track,
weather = stormy_weather
)
# (12) Running of the simulation
sim.run()
# (13) Plotting of the simulation data
plot = Plotter_Speed(sim)
# (14) Plotting of data
plot.plot()
# And then of course there could be other things like saving of data.
So yes, I could put this in one big file, or even a bunch of files in the same directory.
But there are a number of reasons why I don't want to do this.
First, because it's ugly and hard to read and find things in. I'd much rather have it broken apart into separate files which are logically sorted into different directories, and readable.
Second, because if someone wants to add a new, say, windshield type, they've got to find the right spot in the code and modify the original source. I'd prefer to have a "plug-in" system where I just write a new file, stick it in the "windshields" directory, and then have it available from whichever script I choose to import all of my modules from.
So I'd much prefer my code spread over a directory structure like the following:
racecar_sim/
|
+-- run_script.py
|
+-- utils/
| |
| + __init__.py
|
+-- parts/
| |
| +-- materials/
| | |
| | +-- rubber/
| | | |
| | | +-- __init__.py
| | |
| | +-- glass/
| | |
| | +-- __init__.py
| |
| +-- tires/
| | |
| | +-- __init__.py
| |
| +-- windshields/
| |
| +-- __init__.py
|
+-- tracks/
| |
| +-- __init__.py
|
+-- weather/
| |
| +-- __init__.py
|
+-- cars/
| |
| +-- __init__.py
|
+-- simulations/
| |
| + __init__.py
|
+-- parts/
|
+ __init__.py
This would be done in such a way that run_script.py could contain something like this:
from utils import file_io_function_1
from parts.materials.rubber import Material_Rubber_Hard
from parts.windshields import Windshield_Smooth
from cars import Racecar_Fast
# ... ETC ...
from simulations import Simulation_Speed
from plotters import Plotter_Speed
hard_rubber = Material_Rubber_Hard(42)
fast_tires = Tire_Fast(hard_rubber)
smooth_windshield = Windshield_Smooth(0.42)
fast_car = Racecar_Fast(
tires = fast_tires,
windshield = smooth_windshield
)
sim = Simulation_Speed(
car1 = fast_car,
car2 = medium_car,
track = normal_track,
weather = stormy_weather
)
sim.run()
plot = Plotter_Speed(sim)
plot.plot()
However, I'm not sure how to go about doing this, since many of these modules will need to import other ones.
For instance, I want to create a Tire which always uses a particular type of Rubber, so it calls a Rubber child class constructor from its constructor.
Likewise, most of them will want to import the utils module.
It's this "cross branch importing" I'm not sure how to set up, unless I install to site-packages, which I don't want to have to do.
The other option is something like what's spelled out for the test suite here, but it seems a little hackish and I'm not sure it's suited (no pun intended) to something other than a test suite:
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
I'd have to do that in each submodule, with the right path, each new plug-n-play file would have to do the same thing, ... blegh!
I'd also rather avoid as much setup.py stuff as I can. I'd rather the run_script.py in the above example could simply be run without setup of any kind, or at least as minimal as possible.
So what's the correct Pythonic way of doing this? Do I need to completely rethink my idea of how it should be structured? Am I on the right track?
