Handling many optional imports

Question

We have a file with a few different classes which all do the same thing but in different ways. Each of these classes have a couple of imports from external packages.

Our run file then uses the configs (parsed as a dictionary) to find which methodology the user would like i.e. which class to use.

Now, we don't want to force the user to get all the packages if they are only going to use one methodology. So we would like a way of handling conditional imports.

I have seen that some people handle this with a simple:

try:
    import module
except ImportError:
    try:
        import other_module
    except ImportError:
        raise

However, we have a fair few of these imports and daisy-chaining them like this is hard to maintain and clumsy.

I could write something which simply loops through a list of modules until one (or more) imports successfully, but then where should this logic go? In the file containing the classes, or in the run file which imports those classes?

Finally, we could just add the imports to the classes' namespaces:

class EngineOne:

    import module

    def methodology_one(self):

        self.module.function()


class EngineTwo:

    import other_module

    def methodology_two(self):

        self.other_module.function()

This however seems very non-standard and I can't find any real-world examples where people have done this. self.module.function() also feels very odd even if it makes sense to have the imports as class variables.

I've had a brief look through some common data science packages I have installed but can't find any where this sort of thing is used.

I have found a few stackoverflow questions which have briefly mentioned conditional imports such as this one and this one, but none where there are concrete examples which well-describe our case.

Thanks for any help.

Are there so many modules that importing them all by default takes a considerable amount of the overall execution time? If not: importing them and not using them doesn't seem like a bad solution. — GittingGud, Apr 25 '19 at 13:24
I'd agree but execution time isn't the issue for us. It's more that these are large modules with many dependencies which can cause environment issues for users. We want to streamline the installation process for users who don't want everything. — QuantumChris, Apr 25 '19 at 13:27
I think having a list/dictionary with all the moudles which are supposed to be loaded and then looping trough that is the most elegant solution and it *feels* better than the class solution. This logic should be where the modules are needed so in your main execution file. — GittingGud, Apr 25 '19 at 13:35
I'm worried the execution file may become cluttered; might it be better to write this kind of logic in the `__init__.py` file? Although maybe that obfuscates where the modules are coming from. I will try both and hopefully publish an answer. Thanks. — QuantumChris, Apr 25 '19 at 13:40

score 0 · Answer 1 · answered Apr 25 '19 at 14:09

In the end, we've decided to separate the classes for each methodology into separate files, then put them into a directory with an __init__.py. This __init__.py then contains some logic which loops through the imports, catching any import errors.

The structure was:

main_dir
   - __init__.py
   - engines.py
      - class EngineOne
         ...
      - class EngineTwo
         ...
      - class EngineThree
         ...
   - run.py
   - ...

Whereas now, the structure is:

main_dir
   - __init__.py
   - engines_dir
      - __init__.py   # Contains logic for import checking
      - method_one.py
         - class EngineOne
            ...
      - method_two.py
         - class EngineTwo
            ...
      - method_three.py
         - class EngineThree
            ...
   - run.py   # Just imports the engines_dir module
   - ...

We did this for a couple of reasons.

We didn't want the main run file to be cluttered as we added more conditional imports. If a few of our stages of execution have a few options each, the run file is quickly cluttered with looping through imports with respect to configs.
We need our package to be importable as well. This means putting the import checking into the run file would have caused issues when importing directly to elsewhere.

While this is an answer, I'm not convinced it is the answer. There might well be better ways of doing this, but this works well for us. I won't mark the question as solved in case a better reply is given.

Handling many optional imports

1 Answers1