1

I'm trying to find full path to every key in my json

{ "Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder":{ 
  "CoopRoomDifficultySetups": [
    {
          "RoomDifficulties": [
                {
    "Id" : 510,
    "IsEnabled" : 1,
    "AvailableRegions" : [ ],
    "Weight" : 1,
    "MinLevel" : 60,
    "MaxLevel" : 69,
    "MinFriendsLevel" : 50,
    "MaxFriendsLevel" : 99,
    "MinPunishLevel" : 0,
    "MaxPunishLevel" : 0,
    "MinHardCP" : "0",
    "MinCP" : "0",
    "MaxCP" : "0",
    "MaxHardCP" : "0",
    "LowPowerBonus" : 0.5,
    "LowPowerCp" : "0",
    "LowPowerLevel" : 90,
    "Maps" : [ {
      "MapId" : 4,
      "NpcPreset" : "NPCPresetMap5Coop3",
      "TypesLimit" : 1000,
      "TierSpawnProbabilities" : [ {
        "Tier" : 0,
        "SpawnProbability" : 0.6
      }, {
        "Tier" : 1,
        "SpawnProbability" : 0.75
      }, {
        "Tier" : 2,
        "SpawnProbability" : 0.52
      }, {
        "Tier" : 3,
        "SpawnProbability" : 0.6
      } ],
      "ChampionProbabilityTier2" : 0.1,
      "ChampionProbabilityTier3" : 0.08,
      "PlayersWhenMaxProbabilityTier2" : 3,
      "PlayersWhenMaxProbabilityTier3" : 6,
      "NpcLevelMultiplier" : 1.15,
      "MapWeight" : 1,
      "PointsNpcMultiplier" : 0.85,
      "XpNpcMultiplier" : 0.85,
      "ScoreNpcMultiplier" : 0.85,
      "NpcMinLevel" : 63,
      "NpcMaxLevel" : 77
    } ],
    "TimeOfDayMode_Parsable" : 0
}]},[{"foo":"foo"}]]}}

And with that being said I've found a function on stackoverflow to do it, however it doesn't return the exact path that I need to walk through manually to access the values, for example: to access "Id":

json['Assets']['Coop'][0]['Room'][0]['Id'] and then it returns 510

however this function returns the following path:

json['Assets']['Coop']['Room']['Id']

So it looks as if it doesn't read lists like I'd like it to. What's more I've already tried deepmerge library as a solution since my main goal is to read all the values from the json above and then compare it with another json, and when it finds "Id" : 510, then all the values below should be changed

def walktree(tree, at=lambda node: not isinstance(node, dict), prefix=(), 
                flattennode=lambda node:isinstance(node, (list, tuple, set))):
    """
    Traverse a tree, and return a iterator of the paths from the root nodes to the leaf nodes.
    tree: like '{'a':{'b':1,'c':2}}'
    at: a bool function or a int indicates levels num to go down. walktree(tree, at=1) equivalent to tree.items()
    flattennode: a bool function to decide whether to iterate at node value
    """
    if isinstance(at, int):
        isleaf_ = at == 0
        isleaf = lambda v: isleaf_
        at = at - 1
    else:
        isleaf = at
    if isleaf(tree):
        if not flattennode(tree):
            yield (*prefix, tree)
        else:
            for v in tree:
                yield from walktree(v, at, prefix, flattennode=flattennode)
    else:
        for k,v in tree.items():
            yield from walktree(v, at, (*prefix, k), flattennode=flattennode)

3 Answers3

1

Here is a generator which meets your requirements:

def dict_generator(dictionary, previous=None):
    previous = previous[:] if previous else []
    if isinstance(dictionary, dict):
        for key, value in dictionary.items():
            if isinstance(value, dict): 
                for d in dict_generator(value,  previous + [key]):
                    yield d
            elif isinstance(value, list) or isinstance(value, tuple):
                for k,v in enumerate(value):
                    for d in dict_generator(v, previous + [key] + [[k]]):
                        yield d
            else:
                yield previous + [key, value]
    else:
        yield previous + [dictionary]

mydict ={'foo': {'foo1': [1,2,3], 'foo2': 1}}
print(list(dict_generator(mydict)))

Will produce output like this:

[['foo', 'foo1', [0], 1], ['foo', 'foo1', [1], 2], ['foo', 'foo1', [2], 3], ['foo', 'foo2', 1]]
0

Preparation:

jsonpath-ng can parse even such a nested json object very easily. It can be installed by the following command:

pip install --upgrade jsonpath-ng

Code:

import jsonpath_ng as jp

# Create the sample json
data = {"Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder":{ "CoopRoomDifficultySetups": [ { "RoomDifficulties": [ { "Id" : 510, "IsEnabled" : 1, "AvailableRegions" : [ ], "Weight" : 1, "MinLevel" : 60, "MaxLevel" : 69, "MinFriendsLevel" : 50, "MaxFriendsLevel" : 99, "MinPunishLevel" : 0, "MaxPunishLevel" : 0, "MinHardCP" : "0", "MinCP" : "0", "MaxCP" : "0", "MaxHardCP" : "0", "LowPowerBonus" : 0.5, "LowPowerCp" : "0", "LowPowerLevel" : 90, "Maps" : [ { "MapId" : 4, "NpcPreset" : "NPCPresetMap5Coop3", "TypesLimit" : 1000, "TierSpawnProbabilities" : [ { "Tier" : 0, "SpawnProbability" : 0.6 }, { "Tier" : 1, "SpawnProbability" : 0.75 }, { "Tier" : 2, "SpawnProbability" : 0.52 }, { "Tier" : 3, "SpawnProbability" : 0.6 } ], "ChampionProbabilityTier2" : 0.1, "ChampionProbabilityTier3" : 0.08, "PlayersWhenMaxProbabilityTier2" : 3, "PlayersWhenMaxProbabilityTier3" : 6, "NpcLevelMultiplier" : 1.15, "MapWeight" : 1, "PointsNpcMultiplier" : 0.85, "XpNpcMultiplier" : 0.85, "ScoreNpcMultiplier" : 0.85, "NpcMinLevel" : 63, "NpcMaxLevel" : 77 } ], "TimeOfDayMode_Parsable" : 0 }]},[{"foo":"foo"}]]}}

# Define a dictionary
d = {}

# Define an expression to parse the json object
expr = jp.parse('$..*')
for m in expr.find(data):

    # Show a json path
    print(str(m.full_path))

    # Update the dictionary
    d[str(m.full_path)] = m.value

# Show an example value in the dictionary
key = 'Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].NpcMaxLevel'
print(d[key])

Output:

Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Id
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].IsEnabled
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].AvailableRegions
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Weight
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MinLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MaxLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MinFriendsLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MaxFriendsLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MinPunishLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MaxPunishLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MinHardCP
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MinCP
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MaxCP
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].MaxHardCP
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].LowPowerBonus
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].LowPowerCp
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].LowPowerLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].TimeOfDayMode_Parsable
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].MapId
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].NpcPreset
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TypesLimit
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].ChampionProbabilityTier2
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].ChampionProbabilityTier3
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].PlayersWhenMaxProbabilityTier2
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].PlayersWhenMaxProbabilityTier3
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].NpcLevelMultiplier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].MapWeight
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].PointsNpcMultiplier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].XpNpcMultiplier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].ScoreNpcMultiplier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].NpcMinLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].NpcMaxLevel
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[0].Tier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[0].SpawnProbability
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[1].Tier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[1].SpawnProbability
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[2].Tier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[2].SpawnProbability
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[3].Tier
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[0].RoomDifficulties.[0].Maps.[0].TierSpawnProbabilities.[3].SpawnProbability
Assets/CustomGameData/Resources/Configs/RoomDifficulty/RoomDifficultySetupHolder.CoopRoomDifficultySetups.[1].[0].foo

and

77
quasi-human
  • 1,898
  • 1
  • 2
  • 13
  • I think that it might be the solution for me :D however I still need to verify that, anyways thanks <3 – Szymon Kołodziej Feb 25 '22 at 16:05
  • You're welcome. Feel free to ask me any questions like how to use the library. – quasi-human Feb 25 '22 at 16:07
  • I am not quite sure how to use it in practice, since I am a beginner, cause even tho it prints a nice path, when I want to store it in list and access later, it returns some nested structure with "left" and "right" keys, and "field"/"child" in value – Szymon Kołodziej Feb 28 '22 at 10:33
  • I see. It is because this `m.full_path` is not `string` type. So you have to explicitly write `str(m.full_path)`. I'll also update my answer. – quasi-human Feb 28 '22 at 12:04
0

What almost helped me completely is that piece of code https://stackoverflow.com/a/59186999/18018783

def dict_generator(indict, pre=None):
pre = pre[:] if pre else []
if isinstance(indict, dict):
    for key, value in indict.items():
        if isinstance(value, dict):
            for d in dict_generator(value,  pre + [key]):
                yield d
        elif isinstance(value, list) or isinstance(value, tuple):
            for k,v in enumerate(value):
                for d in dict_generator(v, pre + [key] + [k]):
                    yield d
        else:
            yield pre + [key, value]
else:
    yield indict

However still there is one exception when it doesn't work as I wish for example when this structure is given

{
"foo":{
    "foo1": [1,2,3],
    "foo2": 1
}

} the output of dict_generator will be

[1, 2, 3, ['foo', 'foo2', 1]]

so it doesn't give me a path to the key that has a list as an value since my desired output should look that way:

[['foo', 'foo1', [0], 1], ['foo', 'foo2', [1], 2], ['foo', 'foo2, [2], 3], ['foo', 'foo2', 1]]]