Convert a flat tab-delimited file into Json nested structure

Question

I need to convert a flat file in the following format into JSON format. The input and output is shown as under. I have come across this: Create nested JSON from CSV post however, I have an extra information/field level that is used to determine the nested structure in the JSON output. Python pandas does have df.to_json but couldn't find a way to write in the desired output format. Any help will be appreciated.

Input:

name    level   children    size
aaa 7   aaab    2952
aaa 7   aaac    251
aaa 7   aaad    222
aaab    8   xxx 45
aaab    8   xxy 29
aaab    8   xxz 28
aaab    8   xxa 4
aaac    8   ddd 7
aaac    8   xxt 4
aaac    8   xxu 1
aaac    8   xxv 1
ddd 9   ppp 4
ddd 9   qqq 2

Output:

{
 "name": "aaa",
 "size": 5000,
 "children":
    [
        {
        "name": "aaab",
        "size": 2952,
        "children": [
                  {"name": "xxx", "size": 45},
                  {"name": "xxy", "size": 29},
                  {"name": "xxz", "size": 28},
                  {"name": "xxa", "size": 4}
                  ]
        },

        {
        "name": "aaac",
        "size": 251,
        "children": [
                        {
                        "name": "ddd",
                        "size": 7,
                        "children": [
                                     {"name": "ppp", "size": 4},
                                     {"name": "qqq", "size": 2}
                                     ]
                        },
                        {"name": "xxt", "size": 4},
                        {"name": "xxu", "size": 1},
                        {"name": "xxv", "size": 1}
                     ]
        },
        {"name": "aaad","size": 222}
     ]
}

How do you determine that "aaa" has size 5000? – Kevin Jan 22 '14 at 03:38 — Kevin, Jan 22 '14 at 03:38

Kevin · Accepted Answer · 2014-01-22T16:54:38.463

This is fairly simple to do using a two-pass approach: first, construct a node for each individual line. Then, connect each node to its children.

with open("data.txt") as file:
    lines = file.read().split("\n")

#remove header line.
lines = lines[1:]

entries = {}

#create an entry for each child node.
for line in lines:
    name, level, child, size = line.split()
    entries[child] = {"name": child, "size": int(size), "children": []}

#we now have an entry for all nodes that are a child of another node.
#but not for the topmost parent node, so we'll make one for it now.
parents  = set(line.split()[0] for line in lines)
children = set(line.split()[2] for line in lines)
top_parent = (parents - children).pop()
#(just guess the size, since it isn't supplied in the file)
entries[top_parent] = {"name": top_parent, "size": 5000, "children": []}

#hook up each entry to its children
for line in lines:
    name, level, child, size = line.split()
    entries[name]["children"].append(entries[child])

#the nested structure is ready to use!
structure = entries[top_parent]

#display the beautiful result
import pprint
pprint.pprint(structure)

Result:

{'children': [{'children': [{'children': [], 'name': 'xxx', 'size': 45},
                            {'children': [], 'name': 'xxy', 'size': 29},
                            {'children': [], 'name': 'xxz', 'size': 28},
                            {'children': [], 'name': 'xxa', 'size': 4}],
               'name': 'aaab',
               'size': 2952},
              {'children': [{'children': [{'children': [],
                                           'name': 'ppp',
                                           'size': 4},
                                          {'children': [],
                                           'name': 'qqq',
                                           'size': 2}],
                             'name': 'ddd',
                             'size': 7},
                            {'children': [], 'name': 'xxt', 'size': 4},
                            {'children': [], 'name': 'xxu', 'size': 1},
                            {'children': [], 'name': 'xxv', 'size': 1}],
               'name': 'aaac',
               'size': 251},
              {'children': [], 'name': 'aaad', 'size': 222}],
 'name': 'aaa',
 'size': 5000}

Edit: you can remove the children attribute from leaf nodes by using the del statement.

#execute this after the "hook up each entry to its children" section.
#remove "children" from leaf nodes.
for entry in entries.itervalues():
    if not entry["children"]:
        del entry["children"]

Result:

{'children': [{'children': [{'name': 'xxx', 'size': 45},
                            {'name': 'xxy', 'size': 29},
                            {'name': 'xxz', 'size': 28},
                            {'name': 'xxa', 'size': 4}],
               'name': 'aaab',
               'size': 2952},
              {'children': [{'children': [{'name': 'ppp', 'size': 4},
                                          {'name': 'qqq', 'size': 2}],
                             'name': 'ddd',
                             'size': 7},
                            {'name': 'xxt', 'size': 4},
                            {'name': 'xxu', 'size': 1},
                            {'name': 'xxv', 'size': 1}],
               'name': 'aaac',
               'size': 251},
              {'name': 'aaad', 'size': 222}],
 'name': 'aaa',
 'size': 5000}

Thanks @Kevin. The output is not in the required format. Two things here that is undesirable: 1. Items with `'children': []` need not be there in the output and 2. The ordering has changed — user1140126, Jan 22 '14 at 16:47
Do you mean the ordering of the attributes name/size/children? dicts in Python are inherently unordered, so it's just a quirk of my interpreter that it was printed that way. I couldn't change the ordering if I tried. — Kevin, Jan 22 '14 at 16:51

Convert a flat tab-delimited file into Json nested structure

1 Answers1