Convert nltk Tree to JSON representation

Question

I would like to convert the following nltk Tree representation into JSON format:

Desired output:

{
    "scores": {
        "filler": [
            [
                "scores"
            ],
            [
                "for"
            ]
        ],
        "extent": [
            "highest"
        ],
        "team": [
            "India"
        ]
    }
}

Answer 1

It looks like the input tree may contain children with the same name. To support the general case, you could convert each Tree into a dictionary that maps its name to its children list:

from nltk import Tree # $ pip install nltk

def tree2dict(tree):
    return {tree.node: [tree2dict(t)  if isinstance(t, Tree) else t
                        for t in tree]}

Example:

import json
import sys

tree = Tree('scores',
            [Tree('extent', ['highest']),
             Tree('filler',
                  [Tree('filler', ['scores']),
                   Tree('filler', ['for'])]),
             Tree('team', ['India'])])
d = tree2dict(tree)
json.dump(d, sys.stdout, indent=2)

Output:

{
  "scores": [
    {
      "extent": [
        "highest"
      ]
    }, 
    {
      "filler": [
        {
          "filler": [
            "scores"
          ]
        }, 
        {
          "filler": [
            "for"
          ]
        }
      ]
    }, 
    {
      "team": [
        "India"
      ]
    }
  ]
}

Answer 2

Convert Tree to dict and then to JSON.

def tree_to_dict(tree):
    tdict = {}
    for t in tree:
        if isinstance(t, nltk.Tree) and isinstance(t[0], nltk.Tree):
            tdict[t.node] = tree_to_dict(t)
        elif isinstance(t, nltk.Tree):
            tdict[t.node] = t[0]
    return tdict

def dict_to_json(dict):
    return json.dumps(dict)

output_json = dict_to_json({tree.node: tree_to_dict(tree)})

Answer 3

the will covert the tree to a dictionary with the tree labels as the key, then you can convert it into JSON using by using JSON dumps easily

    import nltk.tree.Tree

    def tree_to_dict(tree):
        tree_dict = dict()
        leaves = []
        for subtree in tree:
            if type(subtree) == nltk.tree.Tree:
                tree_dict.update(tree_to_dict(subtree))
            else:
                (expression,tag) = subtree
                leaves.append(expression)
        tree_dict[tree.label()] = " ".join(leaves)

        return tree_dict

Answer 4

A related alternative. For my purposes, I didn't need an exact tree preserved, but instead wanted to extract entities as keys and tokens as lists of values. For the sentence "Tom and Larry play for the Patriots." I wanted the following JSON:

{
  "PERSON": [
    "Tom",
    "Larry"
  ],
  "ORGANIZATION": [
    "Patriots"
  ]
}

This preserves order of tokens (per entity type), while also not "stomping" values set for an entity key. You can reuse the same json.dump code in the other answers to return this dict to json.

from nltk import tag,chunk,tokenize

def prep(sentence):
    return chunk.ne_chunk(tag.pos_tag(tokenize.word_tokenize(sentence)))

t = prep("Tom and Larry play for the Patriots.")

def tree_to_dict(tree):
    tree_dict = dict()
    for st in tree:
        # not everything gets a NE tag,
        # so we can ignore untagged tokens
        # which are stored in tuples
        if isinstance(st, nltk.Tree):
            if st.label() in tree_dict:
                tree_dict[st.label()] = tree_dict[st.label()] + [st[0][0]]
            else:
                tree_dict[st.label()] = [st[0][0]]
    return tree_dict

print(tree_to_dict(t))
# {'PERSON': ['Tom', 'Larry'], 'ORGANIZATION': ['Patriots']}