Auto-generate json paths from given json structure using Python

Question

I am trying to generate auto json paths from given json structure but stuck in the programatic part. Can someone please help out with the idea to take it further?

Below is the code so far i have achieved.

def iterate_dict(dict_data, key, tmp_key):
    for k, v in dict_data.items():

        key = key + tmp_key + '.' + k
        key = key.replace('$$', '$')
        if type(v) is dict:
            tmp_key = key
            key = '$'
            iterate_dict(v, key, tmp_key)
        elif type(v) is list:
            str_encountered = False
            for i in v:
                if type(i) is str:
                    str_encountered = True
                    tmp_key = key
                    break
                tmp_key = key
                key = '$'
                iterate_dict(i, key, tmp_key)
            if str_encountered:
                print(key, v)
                if tmp_key is not None:
                    tmp_key = str(tmp_key)[:-str(k).__len__() - 1]
                key = '$'
        else:
            print(key, v)
            key = '$'


import json
iterate_dict_new(dict(json.loads(d_data)), '$', '')

consider the below json structure

{
    "id": "1",
    "categories": [
        {
            "name": "author",
            "book": "fiction",
            "leaders": [
                 {
                      "ref": ["wiki", "google"],
                      "athlete": {
                            "$ref": "some data"
                       },
                      "data": {
                            "$data": "some other data"
                       }
                 }
             ]
        },
        {
            "name": "dummy name"
        }
    ]
}

Expected output out of python script:

$id = 1
$categories[0].name = author
$categories[0].book = fiction
$categories[0].leaders[0].ref[0] = wiki
$categories[0].leaders[0].ref[1] = google
$categories[0].leaders[0].athlete.$ref = some data
$categories[0].leaders[0].data.$data = some other data
$categories[1].name = dummy name

Current output with above python script:

$.id 1
$$.categories.name author
$$.categories.book fiction
$$$.categories.leaders.ref ["wiki", "google"]
$$$$$.categories.leaders.athlete.$ref some data
$$$$$$.categories.leaders.athlete.data.$data some other data
$$.name dummy name

I am working on a sol, but for recursion to work properly, I think this output line: `$categories[0].leaders[0].ref = ['wiki', 'google']` should be turned into 2 more lines: `...leaders[0].ref[0] = wiki` and `...leaders[0].ref[1] = google` — Joe Iddon, Jan 23 '18 at 17:09
Thanks @JoeIddon, even i am trying out different other ways. Yes, it should be `...leaders[0].ref[0] = wiki` and `...leaders[0].ref[1] = google` i have modified my question accordingly. Other way i could think of is, to convert json to xml structure and auto generate xpaths using sax parser. — user2649233, Jan 23 '18 at 17:28

score 2 · Accepted Answer · answered Jan 23 '18 at 17:39

The following recursive function is similar to yours, but instead of just displaying a dictionary, it can also take a list. This means that if you passed in a dictionary where one of the values was a nested list, then the output would still be correct (printing things like dict.key[3][4] = element).

def disp_paths(it, p='$'):
    for k, v in (it.items() if type(it) is dict else enumerate(it)):
        if type(v) is dict:
            disp_paths(v, '{}.{}'.format(p, k))
        elif type(v) is list:
            for i, e in enumerate(v):
                if type(e) is dict or type(e) is list:
                    disp_paths(e, '{}.{}[{}]'.format(p, k, i))
                else:
                    print('{}.{}[{}] = {}'.format(p, k, i, e))
        else:
            f = '{}.{} = {}' if type(it) is dict else '{}[{}] = {}'
            print(f.format(p, k, v))

which, when ran with your dictionary (disp_paths(d)), gives the expected output of:

$.categories[0].leaders[0].athlete.$ref = some data
$.categories[0].leaders[0].data.$data = some other data
$.categories[0].leaders[0].ref[0] = wiki
$.categories[0].leaders[0].ref[1] = google
$.categories[0].book = fiction
$.categories[0].name = author
$.categories[1].name = dummy name
$.id = 1

Note that this is unfortunately not ordered, but that is unavoidable as dictionaries have no inherent order (they are just sets of key:value pairs)

If you need help understanding my modifications, just drop a comment!

A little modification to handle nested arrays and dict. `'{}.{}'.format(p, k) if type(k) is str else '{}[{}]'.format(p, k)` this needs to be called where ever we are calling the recursion and printing values. — user2649233, Feb 18 '18 at 08:11

Auto-generate json paths from given json structure using Python

1 Answers1