trying to walk a tree as generator, but it seems the walk stops at level1. 'table' and 'of' children are not visited !!
(Could it be because .children is itself generator ? )
def walk(token):
yield token
print token , '->', list(token.children)
for t in token.children :
yield t
walk(t)
In [174]: doc
Out[174]: the table is made of wood
In [175]: list(walk(root(doc)))
made -> [table, is, of]
Out[175]: [made, table, is, of]
In [176]: doc[1]
Out[176]: table
In [178]: list(doc[1].children)
Out[178]: [the]
In [181]: doc[4]
Out[181]: of
In [182]: list(doc[4].children)
Out[182]: [wood]
========================
this question is not duplicate.
As I mentioned .children is also generator.
This seems to work, so far :
def walk(token):
yield token
# print token , '->', list(token.children)
for t in token.children :
for tc in walk(t) : yield tc
In [234]: list(walk(root(doc)))
Out[234]: [made, table, the, is, of, wood]