neatly instantiate class within module

Question

EDIT: I figured it out, would appreciate it if this is marked as NOT a duplicate because it isn't, by keeping __init__.py empty I was effectively creating a submodule with what I wanted to be the main module. Moving some key things over to __init__.py solved my problem.

I have a python module, expecter.py

class expecter():
    def __init__(self, host, username, password):
        # connect to host, setup typical ssh expect things
    def dothingstohost(self):
        self.sendcommandsandthings('etc')

with a setup.py like this (assume project has no subdirectories, simply a single expecter.py file)

from setuptools import setup

setup(
    name='expecter',
    version='1.21',
    packages=['.',],
    install_requires=[
        "pexpect",
    ],
)

and I would like to call it like this

import expecter

connection = expecter('someserver', 'someuser', 'somepass')

However after trying numerous things I keep getting stuck in a situation where after a pip install I have to call it like this connection = expecter.expecter('blahblah'). or pip doesn't like the way I've defined packages and it gets confused. I used to just keep this class alongside my other projects but it's getting larger and I am trying to separate it into its' own project.

This doesn't have to do with pip or setuptools. There is always a distinction between a module and the classes and functions defined in that module. — mkrieger1, Jun 03 '21 at 00:12
Im not sure I understand, what I'm asking to do I can already do if the expecter.py is just sitting next to where I'm working. I just want the same to work as an installed module. The o was a typo — Preston, Jun 03 '21 at 00:16
No, you can't use `import expecter` and then `expecter('someserver', ...)`. You either need to change the import to `from expecter import expecter`, or use the class as `expecter.expecter(...)`. — mkrieger1, Jun 03 '21 at 00:18
You want the module to be callable? Why? Technically possible, but weird. — wim, Jun 03 '21 at 00:19
https://stackoverflow.com/questions/1060796/callable-modules — Hans Musgrave, Jun 03 '21 at 00:21
@mkrieger even when using `from expecter import expecter` I still have to double up when I call it. That's part of why I'm confused. — Preston, Jun 03 '21 at 00:41
@Preston You shouldn't. `from expected import expected` doesn't bind the module to a name the current scope, only the class in the module. — chepner, Jun 03 '21 at 00:44
It would be less confusing if you followed the PEP 8 [naming conventions](https://www.python.org/dev/peps/pep-0008/#naming-conventions) with respect to class and module names — i.e. rename your class `Expecter`. — martineau, Jun 03 '21 at 01:00
I figured it out, would appreciate it if this is marked as NOT a duplicate because it isn't, by keeping \_\_init\_\_.py empty I was effectively creating a submodule with what I wanted to be the main module. Moving some key things over to \_\_init\_\_.py solved my problem. — Preston, Jun 03 '21 at 15:30

Hans Musgrave · Answer 1 · 2021-06-03T01:06:33.357

None of this is really advisable; the ecosystem expects that when they import a module they get that module rather than some other hacked up object. Statements like from expecter import Expecter are not bad Python code. That said, we can make the import system behave as desired.

When you import expecter, the result is the expecter module. To be able to call that you need the module itself to be callable. They aren't generally, so we want to find some way to change that.

Objects are callable if their types implement __call__. Most built-in types can't be readily monkey patched, but in 3.5+ you can hack up modules just like any other object. So, that's what we do :) The solution here was adapted from elsewhere on StackOverflow and swaps out the class of your expecter module with one with __call__ defined, as desired.

import sys

class Expecter:
    pass  # do your ssh voodoo here

class __CallableModuleHack(sys.modules[__name__].__class__):
    def __call__(self, *args, **kwargs):
        return Expecter(*args, **kwargs)

sys.modules[__name__].__class__ = __CallableModuleHack

Another option (inspired by @martineau from the comments) is to just replace the entire module with a phony module with the desired behavior. This should work in at least Python 2.7+.

import sys, types

class Expecter:
    pass  # do your ssh voodoo here

class __FauxModule(types.ModuleType):
    def __init__(self):
        super().__init__(__name__)
        self.__dict__.update(sys.modules[__name__].__dict__)

    def __call__(self, *args, **kwargs):
        return Expecter(*args, **kwargs)

# so that the module containing our class isn't deleted
__ref = sys.modules[__name__]

sys.modules[__name__] = __FauxModule()

Replacing a module entry in `sys.modules` is a very old technique going back to before Python 2.7 (see [my question](https://stackoverflow.com/questions/5365562/why-is-the-value-of-name-changing-after-assignment-to-sys-modules-name) about it for example. — martineau, Jun 03 '21 at 00:37
We're not replacing the entry though; we're modifying the `__class__` attribute on an existing entry. That fails for older python versions for me. Should it not? — Hans Musgrave, Jun 03 '21 at 00:40
Hmm, yes I guess what you're doing is a bit different, you're changing an _attribute_ of an existing `sys.modules` entry. However, I think it's possible to do in older versions, too — it's just not something I've attempted doing very often (if ever). One thing about your answer not mentioned is the fact that it's taking advantage of the fact that the module and the class have exactly the same name, whereas normally class names all start with a capital letter and module names do not (for those following [PEP 8 - Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/)). — martineau, Jun 03 '21 at 00:46
I like your idea better I think. One sec (and as to the name thing, I was just replicating OP to avoid confusion on that front). — Hans Musgrave, Jun 03 '21 at 00:49

neatly instantiate class within module

1 Answers1