I have a bloated nested dictionary A I would like to transform into a concise, less nested dictionary B that hopes to become JSON. From a design perspective this can be done a couple of ways:
The Switch Sifter
One way to do this is to walk A and pass it through a complicated series of switch statements that sifts the data into B's shape. This is bad if A is relatively large since that would need many switch conditions or if one has many different bloated structures like A and need compose many different switch sifters. This is also bad if traces (any breadcrumb path to a deepest nested value) through A change in the future. "The only thing constant is change" though so this would be an issue with most approaches.
The Map Transformer
I speculate that another way to do this is, while walking A, to use a lookup table that maps every possible trace from A onto every new trace that represents and builds B. Said differently, every value from every trace in A is mapped as the value of respective traces in B. Take a look at this example. Making this requires the same knowledge of the A to B transformation logic needed to sift data in the switch sifter but makes that transformation highly readable as a lookup table. The downside to this approach is it's easy to break if A's structure changes and the lookup table doesn't contain the new/changed trace case.
The Map Sifter
I speculate that perhaps a better approach is a hybrid of the prior two that uses a lookup table to transform A into B for all known traces of A but then falls back on switching logic to handle new traces according to some rules known about the way A may change. Of course, all bets are off if the structure of A changes wildly.
The Question
I tend to find examples of and have coded many a Switch Sifter to munge data structures. I've never made Map Transformers or Map Sifters and am wondering if they are effective in the situation where one is trying to simplify many highly nested, redundant structures each of which may have 200+ traces. I'm considering building a Map Sifter (or whatever that's called if it already has a name). Is there a better way of transforming JSON?
I'm working in Python (if you know of a library that could help me out). Below is starter code to play with the idea of a Map Transformer (if it be a viable approach)
import json
def recurse(d, keys=[]):
# walk python object. Thanks to NeilenMarais @ http://stackoverflow.com/questions/8335096/iterate-over-nested-dictionary
if isinstance(d, dict):
for k in d:
for rv in recurse(d[k], keys + [(k,type(d).__name__)]):
yield rv
elif isinstance(d, list):
for i, v in enumerate(d):
for rv in recurse(v, keys + [(i,type(d).__name__)]):
yield rv
else:
yield (keys, d)
def pp_json(json_thing, sort=True, indents=4):
# pretty print jsonesque object
if type(json_thing) is str:
print(json.dumps(json.loads(json_thing), sort_keys=sort, indent=indents))
else:
print(json.dumps(json_thing, sort_keys=sort, indent=indents))
return None
aLongFluffyJson="""{
"anAlrightKey1": "21.5",
"key2": {
"superfluousKey1":{
"keyWhosNameIsTooLong1": "1",
"keyWhosNameIsTooLong2": "2013-08-28",
"anAlrightKey2": "435 PACIFIC AVENUE, 4TH FLOOR"
}
},
"key3": [
{
"keyWhosNameIsTooLong3": "200045",
"anAlrightKey3": "PeoplePlacesThings"
},
{
"keyWhosNameIsTooLong3": "300045",
"anAlrightKey3": "null"
}
],
"key4": {
"key5": {
"key6": "thisValueDoesntMatter",
"key7": "iWishToBeAccessible"
}
}
}"""
aConciseJson="""{
"anAlrightKey1": "21.5",
"key2": {
"keyNowShort1": "true",
"keyNowShort2": "2013-08-28"
},
"key3": [
{
"keyNowShort3": "200045",
"anAlrightKey3": "PeoplePlacesThings"
},
{
"keyNowShort4": "300045"
}
],
"key4": {
"key5": {
"key6": "iWishToBeAccessible",
"key7": "iAlsoWishToBeAccessible",
"anAlrightKey2": "435 PACIFIC AVENUE, 4TH FLOOR"
}
}
}"""
aLongFluffyJson=json.loads(aLongFluffyJson)
aConciseJson=json.loads(aConciseJson)
pp_json(aConciseJson) #See what you're working with
# prepare what could be used to build the lookup table in a Map Transformer
for compound_key, val in recurse(aLongFluffyJson):
print('{}: {}'.format(compound_key, val))
print(" ")
for compound_key, val in recurse(aConciseJson): #this is missing the value type definition
print('{}: {}'.format(compound_key, val))