Is it possible to implement .NET-like attributes in Python?

Question

I'm a big fan of .NETs attributes - pre-defined and user-defined ones. Attributes are classes inherited from Attribute. Mostly everything in .NET (classes, methods, members (properties, fields, enum values)) can be 'decorated'/equipped with attributes. This attributes can be read-back by e.g. the compiler to extract compiler hints or by the user as kind of meta programming.

C# example:

[System.Serializable]
public class SampleClass {
  // Objects of this type can be serialized.
}

VB example:

<System.Serializable()>
Public Class SampleClass
  ' Objects of this type can be serialized.
End Class

In my example Serializable marks a class for serialization. A serializer can now retrieve all members of that class' instances and assemble the instance data to a serialized object. It's also possible so mark single fields to be serialized or not.

A user can get defined attributes from a class with the help of reflection: System.Attribute.GetCustomAttributes(...)

For further reading (MSDN documentation):
- Writing Custom Attributes
- Retrieving Information Stored in Attributes

I'm also a big fan of Python and decorators. Is it possible to implement .NET-like attributes in Python with the help of decorators? What would it look like in Python?

@Serializable
class SampleClass():
  # Objects of this type can be serialized.

Another use case could be the Python argparse library. It's possible to register callback functions, which are called by sub-parsers if the input contains the correct sub-command. A more natural way to define such command line argument syntax could be the usage of decorators.

This question is not about serialization - it's just an usage example.

Answer 1

I have played a bit with class-based decorators and as far as I can tell it's possible to implement .NET-like attributes in Python.

So first let's develop a meaningful use case:
Most of us know the Python argparse command line argument parser. This parser can handle sub-commands like git commit -m "message" where commit is a sub-command and -m <message> is a parameter of this sub-command parser. It's possible to assign a callback function to each sub-command parser.

Python 3.4.2 for Windows has a bug in handling callback functions. It's fixed in 3.5.0 (I haven't tested other 3.4.x versions).

Here is a classic argparse example:

class MyProg():

  def Run(self):
    # create a commandline argument parser
    MainParser = argparse.ArgumentParser(
      description = textwrap.dedent('''This is the User Service Tool.'''),
      formatter_class = argparse.RawDescriptionHelpFormatter,
      add_help=False)

    MainParser.add_argument('-v', '--verbose', dest="verbose", help='print out detailed messages', action='store_const', const=True, default=False)
    MainParser.add_argument('-d', '--debug', dest="debug", help='enable debug mode', action='store_const', const=True, default=False)
    MainParser.set_defaults(func=self.HandleDefault)
    subParsers = MainParser.add_subparsers(help='sub-command help')

    # UserManagement commads
    # create the sub-parser for the "create-user" command
    CreateUserParser = subParsers.add_parser('create-user', help='create-user help')
    CreateUserParser.add_argument(metavar='<Username>', dest="Users", type=str, nargs='+', help='todo help')
    CreateUserParser.set_defaults(func=self.HandleCreateUser)

    # create the sub-parser for the "remove-user" command
    RemoveUserParser = subParsers.add_parser('remove-user', help='remove-user help')
    RemoveUserParser.add_argument(metavar='<UserID>', dest="UserIDs", type=str, nargs='+', help='todo help')
    RemoveUserParser.set_defaults(func=self.HandleRemoveUser)

  def HandleDefault(self, args):
    print("HandleDefault:")

  def HandleCreateUser(self, args):
    print("HandleCreateUser: {0}".format(str(args.Users)))

  def HandleRemoveUser(self, args):
    print("HandleRemoveUser: {0}".format(str(args.UserIDs)))


my = MyProg()
my.Run()

A better and more descriptive solution could look like this:

class MyProg():
  def __init__(self):
    self.BuildParser()
    # ...
  def BuiltParser(self):
    # 1. search self for methods (potential handlers)
    # 2. search this methods for attributes
    # 3. extract Command and Argument attributes
    # 4. create the parser with that provided metadata

  # UserManagement commads
  @CommandAttribute('create-user', help="create-user help")
  @ArgumentAttribute(metavar='<Username>', dest="Users", type=str, nargs='+', help='todo help')
  def HandleCreateUser(self, args):
    print("HandleCreateUser: {0}".format(str(args.Users)))

  @CommandAttribute('remove-user',help="remove-user help")
  @ArgumentAttribute(metavar='<UserID>', dest="UserIDs", type=str, nargs='+', help='todo help')
  def HandleRemoveUser(self, args):
    print("HandleRemoveUser: {0}".format(str(args.UserIDs)))

---

Step 1 - A common Attribute class

So let's develop a common Attribute class, which is also a class-based decorator. This decorator adds himself to a list called __attributes__, which is registered on the function which is to be decorated.

class Attribute():
  AttributesMemberName =  "__attributes__"
  _debug =                False

  def __call__(self, func):
    # inherit attributes and append myself or create a new attributes list
    if (func.__dict__.__contains__(Attribute.AttributesMemberName)):
      func.__dict__[Attribute.AttributesMemberName].append(self)
    else:
      func.__setattr__(Attribute.AttributesMemberName, [self])
    return func

  def __str__(self):
    return self.__name__

  @classmethod
  def GetAttributes(self, method):
    if method.__dict__.__contains__(Attribute.AttributesMemberName):
      attributes = method.__dict__[Attribute.AttributesMemberName]
      if isinstance(attributes, list):
        return [attribute for attribute in attributes if isinstance(attribute, self)]
    return list()

Step 2 - User defined attributes

Now we can create custom attributes which inherit the basic decorative functionality from Attribute. I'll declare 3 attributes:

• DefaultAttribute - If no sub-command parser recognizes a command, this decorated method will be the fallback handler.
• CommandAttribute - Define a sub-command and register the decorated function as a callback.
• ArgumentAttribute - Add parameters to the sub-command parser.

class DefaultAttribute(Attribute):
  __handler = None

  def __call__(self, func):
    self.__handler = func
    return super().__call__(func)

  @property
  def Handler(self):
    return self.__handler

class CommandAttribute(Attribute):
  __command = ""
  __handler = None
  __kwargs =  None

  def __init__(self, command, **kwargs):
    super().__init__()
    self.__command =  command
    self.__kwargs =   kwargs

  def __call__(self, func):
    self.__handler = func
    return super().__call__(func)

  @property
  def Command(self):
    return self.__command

  @property
  def Handler(self):
    return self.__handler

  @property
  def KWArgs(self):
    return self.__kwargs

class ArgumentAttribute(Attribute):
  __args =   None
  __kwargs = None

  def __init__(self, *args, **kwargs):
    super().__init__()
    self.__args =   args
    self.__kwargs = kwargs

  @property
  def Args(self):
    return self.__args

  @property
  def KWArgs(self):
    return self.__kwargs

Step 3 - Building a helper mixin class to handle attributes on methods

To ease the work with attributes I implemented a AttributeHelperMixin class, that can:

• retrieve all methods of a class
• check if a method has attributes and
• return a list of attributes on a given method.

class AttributeHelperMixin():
  def GetMethods(self):
    return {funcname: func
            for funcname, func in self.__class__.__dict__.items()
            if hasattr(func, '__dict__')
           }.items()

  def HasAttribute(self, method):
    if method.__dict__.__contains__(Attribute.AttributesMemberName):
      attributeList = method.__dict__[Attribute.AttributesMemberName]
      return (isinstance(attributeList, list) and (len(attributeList) != 0))
    else:
      return False

  def GetAttributes(self, method):
    if method.__dict__.__contains__(Attribute.AttributesMemberName):
      attributeList = method.__dict__[Attribute.AttributesMemberName]
      if isinstance(attributeList, list):
        return attributeList
    return list()

Step 4 - Build an application class

Now it's time to build an application class that inherits from MyBase and from ArgParseMixin. I'll discuss ArgParseMixin later. The class has a normal constructor, which calls both base-class constructors. It also adds 2 arguments for verbose and debug to the main-parser. All callback handlers are decorated with the new Attributes.

class MyBase():
  def __init__(self):
    pass

class prog(MyBase, ArgParseMixin):
  def __init__(self):
    import argparse
    import textwrap

    # call constructor of the main interitance tree
    MyBase.__init__(self)

    # Call the constructor of the ArgParseMixin
    ArgParseMixin.__init__(self,
      description = textwrap.dedent('''\
        This is the Admin Service Tool.
        '''),
      formatter_class = argparse.RawDescriptionHelpFormatter,
      add_help=False)

    self.MainParser.add_argument('-v', '--verbose',  dest="verbose",  help='print out detailed messages',  action='store_const', const=True, default=False)
    self.MainParser.add_argument('-d', '--debug',    dest="debug",    help='enable debug mode',            action='store_const', const=True, default=False)

  def Run(self):
    ArgParseMixin.Run(self)

  @DefaultAttribute()
  def HandleDefault(self, args):
    print("DefaultHandler: verbose={0}  debug={1}".format(str(args.verbose), str(args.debug)))

  @CommandAttribute("create-user", help="my new command")
  @ArgumentAttribute(metavar='<Username>', dest="Users", type=str, help='todo help')
  def HandleCreateUser(self, args):
    print("HandleCreateUser: {0}".format(str(args.Users)))

  @CommandAttribute("remove-user", help="my new command")
  @ArgumentAttribute(metavar='<UserID>', dest="UserIDs", type=str, help='todo help')
  def HandleRemoveUser(self, args):
    print("HandleRemoveUser: {0}".format(str(args.UserIDs)))

p = prog()
p.Run()

Step 5 - The ArgParseMixin helper class.

This class constructs the argparse based parser with the provided data from attributes. The parsing process is invoked by Run().

class ArgParseMixin(AttributeHelperMixin):
  __mainParser = None
  __subParser =  None
  __subParsers = {}

  def __init__(self, **kwargs):
    super().__init__()

    # create a commandline argument parser
    import argparse
    self.__mainParser = argparse.ArgumentParser(**kwargs)
    self.__subParser =  self.__mainParser.add_subparsers(help='sub-command help')

    for funcname,func in self.GetMethods():
      defAttributes = DefaultAttribute.GetAttributes(func)
      if (len(defAttributes) != 0):
        defAttribute = defAttributes[0]
        self.__mainParser.set_defaults(func=defAttribute.Handler)
        continue

      cmdAttributes = CommandAttribute.GetAttributes(func)
      if (len(cmdAttributes) != 0):
        cmdAttribute = cmdAttributes[0]
        subParser = self.__subParser.add_parser(cmdAttribute.Command, **(cmdAttribute.KWArgs))
        subParser.set_defaults(func=cmdAttribute.Handler)

        for argAttribute in ArgumentAttribute.GetAttributes(func):
          subParser.add_argument(*(argAttribute.Args), **(argAttribute.KWArgs))

        self.__subParsers[cmdAttribute.Command] = subParser
        continue

  def Run(self):
    # parse command line options and process splitted arguments in callback functions
    args = self.__mainParser.parse_args()
    # because func is a function (unbound to an object), it MUST be called with self as a first parameter
    args.func(self, args)

  @property
  def MainParser(self):
    return self.__mainParser

  @property
  def SubParsers(self):
    return self.__subParsers

---

I'll provide my code and examples on GitHub as pyAttribute repository.

Answer 2

Given this Serializable attribute, you probably want to have a simple solution.

class C:
    b = 2

    def __init__(self):
        self.a = 1

    def f(self):
        pass


>>> c = C()
>>> c.__dict__
{'a': 1}

80% of work is already done in __dict__ magic attribute that is available for every object. You probably want to have a class-level list of serializable members and make use of __getattribute__ magic methid for modifying what your __dict__ attribute will return for your class.

The same applies to the rest of C# attributes you want to port. I don't think there's a general way to port a random attribute to decorator syntax without writing a ton of code. So for the sake of simplicity, my suggestion is not to stick to decorators and look for short and simple ways.