How to read order of declared members in Python class with use reflection or parsing (metaclass replace is forbidden)?

Question

Important notes to avoid too fast/invalid duplicate tagging - please read it before answer.

Please do not suggest any solution changing original class code - not changing code reflection and parsing is allowed.

How to read class attributes in the same order as declared? is solution - it requires to replace meta class in all classes and add overhead - definitely it is not reflection use.
Consider that I can not or do not want change code to scan class members order. Classes can have or has unknown metaclasses already - it is not possible to just add metaclass or add performance overhead without reason.
Only reflection can be used or parsing files.

I want to avoid writing parser and read class attributes in order of declaration.

How it is possible with use reflection (and simple parsing) in Python?

Let me give some example:

class A(object):
  b = 1
  a = 1
  c = 1

dir(A) give alphabetic order but required is declaration order. How to do it - please help?

If I can read module from class 'A.__module__' I can easily parse code to get order with high accuracy - I can also read some order of base classes of class. I hope that I am not the first person doing this thing. — Chameleon, Jun 21 '14 at 09:33
`inspect.getsourcelines(A)` and parse that, `inspect.getsource(A).split(":")[1]` will print `b=1,a=1,c=1` — Padraic Cunningham, Jun 21 '14 at 09:38
Looks that is the best answer - simple and good - I will check it :) Add you answer I will score it - if check will be successful. — Chameleon, Jun 21 '14 at 09:40

Martijn Pieters · Accepted Answer · 2014-06-21T17:40:45.767

You'll have to resort to parsing. You don't need to write a parser here; the ast module can do this for you.

Parse the source with ast.parse(), then walk the resulting tree:

class ClassOrder(ast.NodeVisitor):
    identifiers = None
    def visit_ClassDef(self, node):
        self.identifiers = []
        for child in node.body:
            if isinstance(child, ast.Assign):
                for target in child.targets:
                    self.visit(target)
            elif isinstance(child, ast.FunctionDef):
                self.identifiers.append(child.name)
    def visit_Name(self, node):
        if self.identifiers is not None:
            self.identifiers.append(node.id)

tree = ast.parse(sourcecode)
order = ClassOrder()
order.visit(tree)
print order.identifiers

prints out the order of assignments and methods in all class definitions.

Demo:

>>> sourcecode = '''\
... class A(object):
...   b = 1
...   a = 1
...   c = 1
... '''
>>> tree = ast.parse(sourcecode)
>>> order = ClassOrder()
>>> order.visit(tree)
>>> print order.identifiers
['b', 'a', 'c']
>>> tree = ast.parse(inspect.getsource(ast.NodeVisitor))
>>> order = ClassOrder()
>>> order.visit(tree)
>>> print order.identifiers
['visit', 'generic_visit']

Bundled up as a function, using inspect.getsource():

import inspect

def get_identifiers(obj):
    source = inspect.getsource(obj)
    tree = ast.parse(source)
    order = ClassOrder()
    order.visit(tree)
    return order.identifiers

inspect.getsource() can handle any object for which the source is available. For classes, it'll extract just the source block defining that class, if available.

Chameleon · Answer 2 · 2014-06-21T17:42:52.803

It is alternative inspire by @Martijn Pieters - this production code using different approach in scanning.

It allow scan __main__ what is problem for inspect.getsource(obj).

It collect also types for each name and function - order scan will allow filter more attributes, functions and extract more information.

import ast
import sys
import codecs
import pprint

class A(object):
  b = 1
  a = 1
  c = 1
  e, f, g = (1, 2, 3)
  z = x = 1
  ((a1, b1), (c1, d1)) = ((1, 2), (1, 2))

class B(A):
  pass

class ClassOrderVisitor(ast.NodeVisitor):
  def __init__(self):
    self.classes = {}

  def __parseTuple(self, astTuple, fields):
    for element in astTuple.elts:
      if isinstance(element, ast.Name):
        fields.append((element.id, element))
      elif isinstance(element, ast.Tuple):
        self.__parseTuple(element, fields)
      else:
        raise NotImplementedError()

  def visit_ClassDef(self, node):
    fields = []
    for field in ast.iter_fields(node):
      # class name
      if field[0] == 'name':
        className = field[1]
        self.classes[className] = fields
      # class body
      elif field[0] == 'body':
        for bodyItem in field[1]:
          if isinstance(bodyItem, ast.Assign):
            for target in bodyItem.targets:
              if isinstance(target, ast.Name):
                fields.append((target.id, target))
              elif isinstance(target, ast.Tuple):
                self.__parseTuple(target, fields)
              else:
                raise NotImplementedError()

          elif isinstance(bodyItem, ast.FunctionDef):
            fields.append((bodyItem.name, bodyItem))

# this file is named ast_parser.py not using inspect.getsource(obj)
# since problem with __main__ scan
def scanOrder(fileName):
  with codecs.open(fileName, encoding = 'utf8') as sourceFile:
    sourceCode = sourceFile.read()
  codeTree = ast.parse(sourceCode, fileName)
  classOrderVisitor = ClassOrderVisitor()
  classOrderVisitor.visit(codeTree)
  return classOrderVisitor.classes

# run
pprint.pprint(scanOrder('ast_parser.py'))
print [x for x in dir(A) if not x.startswith('__') and not x.endswith('__')]

Output:

{'A': [('b', <_ast.Name object at 0x01375E70>),
       ('a', <_ast.Name object at 0x01375ED0>),
       ('c', <_ast.Name object at 0x01375F30>),
       ('e', <_ast.Name object at 0x01375FB0>),
       ('f', <_ast.Name object at 0x01375FD0>),
       ('g', <_ast.Name object at 0x01375FF0>),
       ('z', <_ast.Name object at 0x0137B0D0>),
       ('x', <_ast.Name object at 0x0137B0F0>),
       ('a1', <_ast.Name object at 0x0137B190>),
       ('b1', <_ast.Name object at 0x0137B1B0>),
       ('c1', <_ast.Name object at 0x0137B1F0>),
       ('d1', <_ast.Name object at 0x0137B210>)],
 'B': [],
 'ClassOrderVisitor': [('__init__', <_ast.FunctionDef object at 0x0137B3D0>),
                       ('__parseTuple',
                        <_ast.FunctionDef object at 0x0137B4F0>),
                       ('visit_ClassDef',
                        <_ast.FunctionDef object at 0x0137BA10>)]}
['a', 'a1', 'b', 'b1', 'c', 'c1', 'd1', 'e', 'f', 'g', 'x', 'z']

How to read order of declared members in Python class with use reflection or parsing (metaclass replace is forbidden)?

2 Answers2