Generate (multilevel) flare.json data format from flat json

Question

I have a flat json file structure like:

[
 { "name" : "ABC", "parent":"DEF", "relation": "ghi", "depth": 1 },
 { "name" : "DEF", "parent":"null", "relation": "null", "depth": 0 },
 { "name" : "new_name", "parent":"ABC", "relation": "rel", "depth": 2 }
 ....
 ....
 ]

And what I want is a nested file structure like:

[ 
 {
   "name": "DEF",
   "parent": "null",
   "relation": "null",
   "children": [
                 { "name": "ABC",
                   "parent": "DEF",
                   "relation": "ghi",
                   "children": [
                                 "name": "new_name",
                                 ...
                                 "children": []
                               ]
                 }
               ]
  }
 ]

There is no limit on how many levels deep it should go. The current max I have is 30. There is no limit on the number of children a node can have. Eg. The root node has all the remaining as its children.

What I have tried till now?

• Read about d3.nest() and how it is able to nest but not perfectly. https://groups.google.com/forum/?fromgroups=#!topic/d3-js/L3UeeUnNHO8/discussion

• Wrote a python script for this, but it gets stuck with the null values and also since the data is not bounded (It increases everyday in double figures) so it is very very slow.

• I tried the force directed layout and it worked very well but I want to add another layout that makes the visualization easy.

• I could go with some other python scripts posted but they do not seem to carry forward any other information than "name" and "children".

• I read this: http://blog.pixelingene.com/2011/07/building-a-tree-diagram-in-d3-js/ but they too have the right format data in the first place. What I intend to create is http://bl.ocks.org/mbostock/4339083.

The source of the data is MS SQL Server database which I am fetching and parsing through python. Kindly help! i have been stuck at this for the past 2 weeks.

Thanks

Answer 1

Here's one implementation, in Javascript: http://jsfiddle.net/9FqKS/

You start by creating a name-based map for easy lookup. There are a few different ways to do this - in this case, I use a .reduce method, which starts with an empty object and iterates over the data array, adding an entry for each node:

// create a {name: node} map
var dataMap = data.reduce(function(map, node) {
    map[node.name] = node;
    return map;
}, {});

This is equivalent to:

var dataMap = {};
data.forEach(function(node) {
    dataMap[node.name] = node;
});

(I sometimes think the reduce is more elegant.) Then iteratively add each child to its parents, or to the root array if no parent is found:

// create the tree array
var tree = [];
data.forEach(function(node) {
    // find parent
    var parent = dataMap[node.parent];
    if (parent) {
        // create child array if it doesn't exist
        (parent.children || (parent.children = []))
            // add node to parent's child array
            .push(node);
    } else {
        // parent is null or missing
        tree.push(node);
    }
});

Unless your tree is enormous, I don't think this should be too expensive, so you ought to be able to do it on the client side (if you can't, you might have too much data to easily display in any case).