Python: Loop Nested JSON with unknown depth

Question

I have a JSON response coming back which does a general structure, however, they can be N number of nested results ('queues'). I'm trying to understand how I can loop through all of the nested results which could go extremely deep. I included a link to trinket with the long JSON response. Basically, there are N number of leaf queues that can have no limit, so how can i keep looping deeper? Right now i get stuck as 2 levels down.

import json

with open('sample_response.json') as data_file:    
    theJSON = json.load(data_file)

queuesJson = theJSON['scheduler']['schedulerInfo']['queues']['queue']

def get_leaf_queue_info (childQueue):
    for queue in childQueue:
        print ('nested +' + queue.get('queueName'))

for queue in queuesJson:
    print (queue.get('queueName'))
    if (len(queue['queues']['queue']) > 1):
        get_leaf_queue_info(queue['queues']['queue'])

Link to Code and Sample JSON Response I'm parsing

To handle arbitrarily deep nesting you need to use recursion. Take a look at my code [here](https://stackoverflow.com/a/41778581/4014959), you can probably adapt it to your needs. — PM 2Ring, Nov 08 '17 at 12:26

Tomalak · Accepted Answer · 2017-11-08T13:31:37.363

You have a nested structure - effectively a tree, consisting of nodes that can have child nodes and your task is to:

traverse the tree from the root,
do some work for each node,
traverse each child node.

This is the most classic case for recursion.

In generic terms we cold define a recursive function traverse() like so:

def traverse(node):
    do_work(node)
    for child in children_of(node):
        traverse(child)

This would visit every node and do some work on it. Since it is so generic, it would work on any tree, assuming that do_work() and children_of() are defined.

In order to remove that implicit dependency and make the traverse() function truly generic, let's make it explicit:

def traverse(node, do_work, children_of):
    do_work(node)
    for child in children_of(node):
        traverse(child)

Now for your case, a function that gets the children for any node would look like this:

def sub_queues(queue):
    return queue.get('queues', {}).get('queue', [])

And a function that does some work could look like this:

def print_name(queue):
    print(queue.get('queueName'))

And we could call it like this:

root = data['scheduler']['schedulerInfo']
traverse(root, print_name, sub_queues)

A traverse() function that informs the worker of the current recursion depth (useful if you want to print something with indentation, for example) and cancels the recursion when the worker function returns False could look like this:

def traverse(node, do_work, children_of):
    continue_working = True

    def work(node, depth=0):
        if continue_working:
            result = do_work(node, depth=depth)
            if continue_working and result == False:
                continue_working = False
                return
            for child in children_of(node):
                work(child, depth + 1)

    work(node)

Python: Loop Nested JSON with unknown depth

1 Answers1