3

Trying to use transitions package as per examples provided here https://github.com/pytransitions/transitions

For some reason, neither of the two approaches shown below provide typing suggestions for registered evaporate() trigger (at least in PyCharm 2019.1.2 for Windows x64)

At the same time, these triggers can still be used.

What can be done to have these triggers suggested as I type?

class Matter(Machine):
    def say_hello(self): print("hello, new state!")
    def say_goodbye(self): print("goodbye, old state!")

    def __init__(self):
        states = ['solid', 'liquid', 'gas']
        Machine.__init__(self, states=states, initial='liquid')
        self.add_transition('melt', 'solid', 'liquid')

testmatter= Matter()
testmatter.add_transition('evaporate', 'liquid', 'gas')
testmatter.evaporate()
Out: True

testmatter.get_model_state(testmatter)
Out: <State('gas')@14748976>
class Matter2():
    pass
testmatter2 = Matter2()
machine  = Machine(model=testmatter2, states=['solid', 'liquid', 'gas', 'plasma'], initial='liquid')
machine.add_transition('evaporate', 'liquid', 'gas')

testmatter2.evaporate()
Out: True

1 Answers1

6

transitions adds triggers at runtime to the model (Matter) instance. This cannot be predicted by IDEs before the initialization code is actually executed. Imho, this is the biggest disadvantage of the way in which transitions works (but again imho, it is also its strength when dealing with dynamic state machines or state machines created/received during runtime but that's another story)

If you use an interactive shell with code completion (ipython), you will see that evaporate (based on __dir__ calls to the model) will be suggested:

from transitions import Machine

class Model:
    pass

model = Model()
>>> model.e  # TAB -> nothing

# model will be decorated during machine initialization
machine = Machine(model, states=['A', 'B'], 
                  transitions=[['evaporate', 'A', 'B']], initial='A')

>>> model.e  # TAB -> completion! 

But I assume that's not how you plan to code. So how can we give hints to the introspection?

Easiest solution: Use a docstring for your model to announce triggers.

from transitions import Machine

class Model:
    """My dynamically extended Model
    Attributes:
        evaporate(callable): dynamically added method
    """

model = Model()
# [1]
machine = Machine(model, states=['A', 'B'],
                  transitions=[['evaporate', 'A', 'B']], initial='A')
model.eva  # code completion! will also suggest 'evaporate' before it was added at [1]

The problem here is that the IDE will rely on the docstring to be correct. So when the docstring method (masked as attribute) is calles evaparate, it will always suggests that even though you later add evaporate.

Use pyi files and PEP484 (PyCharm workaround)

Unfortunately, PyCharm does not consider attributes in docstrings for code completion as you correctly pointed out (see this discussion for more details). We need to use another approach. We can create so called pyi files to provide hints to PyCharm. Those files are named identically to their .py counterparts but are solely used for IDEs and other tools and must not be imported (see this post). Let's create a file called sandbox.pyi

# sandbox.pyi

class Model:
    evaporate = None  # type: callable

And now let's create the actual code file sandbox.py (I don't name my playground files 'test' because that always startles pytest...)

# sandbox.py
from transitions import Machine

class Model:
    pass

## Having the type hints right here would enable code completion BUT
## would prevent transitions to decorate the model as it does not override 
## already defined model attributes and methods.
# class Model:
#     evaporate = None  # type: callable

model = Model()
# machine initialization
model.ev  # code completion 

Code completion with pyi files

This way you have code completion AND transitions will correctly decorate the model. The downside is that you have another file to worry about which might clutter your project.

If you want to generate pyi files automatically, you could have a look at stubgen or extend Machine to generate event stubs of models for you.

from transitions import Machine

class Model:
    pass


class PyiMachine(Machine):

    def generate_pyi(self, filename):
        with open(f'{filename}.pyi', 'w') as f:
            for model in self.models:
                f.write(f'class {model.__class__.__name__}:\n')
                for event in self.events:
                    f.write(f'    def {event}(self, *args, **kwargs) -> bool: pass\n')
                f.write('\n\n')


model = Model()
machine = PyiMachine(model, states=['A', 'B'],
                     transitions=[['evaporate', 'A', 'B']], initial='A')
machine.generate_pyi('sandbox')
# PyCharm can now correctly infer the type of success
success = model.evaporate()
model.to_A()  # A dynamically added method which is now visible thanks to the pyi file

Alternative: Generate machine configurations FROM docstrings

A similar issue has been already discussed in the issue tracker of transitions (see https://github.com/pytransitions/transitions/issues/383). You could also generate the machine configuration from the model's docstring:

import transitions
import inspect
import re


class DocMachine(transitions.Machine):
    """Parses states and transitions from model definitions"""

    # checks for 'attribute:value' pairs (including [arrays]) in docstrings
    re_pattern = re.compile(r"(\w+):\s*\[?([^\]\n]+)\]?")

    def __init__(self, model, *args, **kwargs):
        conf = {k: v for k, v in self.re_pattern.findall(model.__doc__, re.MULTILINE)}
        if 'states' not in kwargs:
            kwargs['states'] = [x.strip() for x in conf.get('states', []).split(',')]
        if 'initial' not in kwargs and 'initial' in conf:
            kwargs['initial'] = conf['initial'].strip()
        super(DocMachine, self).__init__(model, *args, **kwargs)
        for name, method in inspect.getmembers(model, predicate=inspect.ismethod):
            doc = method.__doc__ if method.__doc__ else ""
            conf = {k: v for k, v in self.re_pattern.findall(doc, re.MULTILINE)}
            # if docstring contains "source:" we assume it is a trigger definition
            if "source" not in conf:  
                continue
            else:
                conf['source'] = [s.strip() for s in conf['source'].split(', ')]
                conf['source'] = conf['source'][0] if len(conf['source']) == 1 else conf['source']
            if "dest" not in conf:
                conf['dest'] = None
            else:
                conf['dest'] = conf['dest'].strip()
            self.add_transition(trigger=name, **conf)

    # override safeguard which usually prevents accidental overrides
    def _checked_assignment(self, model, name, func):
        setattr(model, name, func)


class Model:
    """A state machine model
    states: [A, B]
    initial: A
    """

    def go(self):
        """processes information
        source: A
        dest: B
        conditions: always_true
        """

    def cycle(self):
        """an internal transition which will not exit the current state
        source: *
        """

    def always_true(self):
        """returns True... always"""
        return True

    def on_exit_B(self):  # no docstring
        raise RuntimeError("We left B. This should not happen!")


m = Model()
machine = DocMachine(m)
assert m.is_A()
m.go()
assert m.is_B()
m.cycle()
try:
    m.go()  # this will raise a MachineError since go is not defined for state B
    assert False
except transitions.MachineError:
    pass

This is a very simple docstring-to-machine-configration parser which does not take care of all eventualities that could be part of a docstring. It assumes that every method with a docstring containing ("source: " ) is supposed to be a trigger. It does however also approaches the issue of documentation. Using such a machine would make sure that at least some documentation for the developed machine exists.

aleneum
  • 2,083
  • 12
  • 29
  • 1
    Thank you for the greate module! I'm trying to follow the docstring approach and found that PyCharm still doesn't do it right [in the editor](http://prntscr.com/r1qtfk) (and it seems to be a [known issue](https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000665110-auto-completion-for-dynamic-module-attributes-in-python)) but does autocomplete in [the Python Console](http://prntscr.com/r1qxbf) – Pavel Paltsev Feb 13 '20 at 12:11
  • At the same time, I've got exactly the same results even if I remove the docstring attribute part. I.e. using ModelOld, model_old and machine_old: [in the editor](http://prntscr.com/r1r02j), [in the Python Console](http://prntscr.com/r1r0k2) model.__dir__() == model_old.__dir__() evaluates to True. So I'm kind of lost on what's the difference then. – Pavel Paltsev Feb 13 '20 at 12:13
  • 1
    @PavelPaltsev: you are right, PyCharm does not consider docstrings unfortunately. I added an example based on 'pyi' files that are also mentioned in the post you referenced. – aleneum Feb 13 '20 at 14:35
  • Thanks for expanding on the answer. Yes, the incompatibility between docstring approach and PyCharm is quite unfortunate because it seems the most straightforward way.. And while using `.pyi` file, do you think there's a way to use a static list with triggers' names to create all the attributes, or I need to specify each trigger/attribute in a separate line as `evaporate = None` (so far I'm [said](https://stackoverflow.com/questions/60209508/type-annotations-with-pyi-file-in-pycharm-doesnt-work-for-dynamic-attributes#comment106498651_60209508) that's the only way to go) – Pavel Paltsev Feb 13 '20 at 14:56
  • 1
    @PavelPaltsev: As far as I know pyi files need to follow a certain format. If instantiating your machines and models is possible you can generate pyi files dynamically by writing pyi files based on a) the output of the inspect module or by extending machine with a generating method. I extended the pyi part with an initial suggestion about how such a method could look like. – aleneum Feb 13 '20 at 15:34