How to create nested dictionaries with duplicate keys in python

Question

I want to create data structure with nested dictionaries and duplicate keys. A detailed example is:

data['State1']['Landon']['abc Area'] = 'BOB'
data['State1']['Landon']['abc Area'] = 'SAM'
data['State1']['Landon']['xyz Area'] = 'John'
data['State2']['New York']['hjk Area'] = 'Ricky'

for z in data['State1'].keys() , 
# I should get list ['Landon', 'Landon', 'Landon']
for y in data['State1']['Landon'].keys() , 
# I should get list ['abc Area', 'abc Area', 'xyz Area']

Currently to store the data I have used extra counter key

data = Autovivification()  
data[state][city][area][counter] = ID

But while parsing total entries (duplicates as well) of City/Area, I have to use nested loops till counter key.

for city in data['State1'].keys():
  for area in data['State1'][city].keys():
    for counter in data['State1'][city][area].keys():
     for temp in data['State1'][city][area][counter].values():
         cityList.append(city)
         areaList.append(area)

For nested dictionaries, I found the following code posted by nosklo

class AutoVivification(dict):  
    """Implementation of perl's autovivification feature."""  
    def __getitem__(self, item):                
         try:  
            return dict.__getitem__(self, item)  
        except KeyError:   
            value = self[item] = type(self)()  
            return value

and for dictionary with duplicate keys, I found code posted by Scorpil

class Dictlist(dict):  
    def __setitem__(self, key, value):  
        try:   
            self[key]   
        except KeyError:   
            super(Dictlist, self).__setitem__(key, [])   
        self[key].append(value)

how to merge Autovivification and Duplicate class code? or is there any other pythonic way to handle such scenario?

Can you explain, why do you need duplicate keys? Is it ok if 'Area' will be list, where you can add multiple ID? — Mikhail Gerasimov, Mar 22 '16 at 13:12
@germn : I have edited the question with more explanation , can you please check — Aniketan, Mar 25 '16 at 07:08
I updated answer adding new `items_in` function that would return list you need for given data. I'm sure it would be easier and clearer to implement functions to work with dict/list structure, then trying to implement your new structure. — Mikhail Gerasimov, Mar 25 '16 at 07:50

Mikhail Gerasimov · Accepted Answer · 2016-03-25T07:47:00.743

One more example using defaultdict:

from collections import defaultdict


data = defaultdict(  # State
    lambda: defaultdict(  # City
        lambda: defaultdict(list)  # Area
    )
)


data['State']['City']['Area'].append('area 1')
data['State']['City']['Area'].append('area 2')
data['State']['City']['Area'].append('area 2')


areas = data['State']['City']['Area']
print(areas)  # ['area 1', 'area 2', 'area 2']

total = len(areas)
print(total)  # 3

How to get list of items you want, using this solution:

data['State1']['Landon']['abc Area'].append('BOB')
data['State1']['Landon']['abc Area'].append('SAM')
data['State1']['Landon']['xyz Area'].append('John')
data['State2']['New York']['hjk Area'].append('Ricky')


def items_in(d):
    res = []
    if isinstance(d, list):
        res.extend(d)
    elif isinstance(d, dict):
        for k, v in d.items():
            res.extend([k] * len(items_in(v)))
    else:
        raise ValueError('Unknown data')
    return res


print(items_in(data['State1']))  # ['Landon', 'Landon', 'Landon']
print(items_in(data['State1']['Landon']))  # ['xyz Area', 'abc Area', 'abc Area']
print(items_in(data['State1']['Landon']['abc Area']))  # ['BOB', 'SAM']
print(items_in(data['State1']['Landon']['xyz Area']))  # ['John']

print(items_in(data['State2']))  # ['New York']
print(items_in(data['State2']['New York']))  # ['hjk Area']

score 1 · Answer 2 · answered Mar 22 '16 at 13:54

1

One simple way would be to make it a list and then just add every new key to the list:

Data['State']['City']['Area'] = []
Data['State']['City']['Area'].append( ID )

answered Mar 22 '16 at 13:54

cptPH

2,401
1
29
35

score 0 · Answer 3 · answered Mar 22 '16 at 16:06

0

You could replace the AutoVivication class with one that auto-vivificates Dictlists instead of dicts:

class AutoVivificationDL(Dictlist):  
    """Implementation of perl's autovivification feature."""  
    def __getitem__(self, item):                
         try:  
            return dict.__getitem__(self, item)  
        except KeyError:   
            value = self[item] = type(self)()  
            return value

answered Mar 22 '16 at 16:06

egpbos

502
4
15

Thanks, I have edited the question with more explanation , can you please check ? , when I used your code 'RuntimeError: maximum recursion depth exceeded while calling a Python object ' is observed. – Aniketan Mar 25 '16 at 07:10
Hm, hadn't thought of that. I think this is Python telling you that this whole AutoVivification thing is not Pythonic :) You're probably better off using a combination of `defaultdict` and regular lists instead of "duplicate entries". If you haven't done so yet, you might also try to reconsider whether you need such a super-flexible datastructure at all, or whether you know enough about your data in advance to just build a nested dict with a predetermined depth. – egpbos Mar 25 '16 at 17:33

JL Peyret · Answer 4 · 2016-03-22T16:21:22.820

Data = {}


values = [
    dict(State="CA", City="San Francisco", Area="North", Id="customer1"),
    dict(State="CA", City="San Francisco", Area="Embarcadero", Id="customer1"),
    dict(State="CA", City="San Francisco", Area="North", Id="customer2"),

]

for v in values:
    #grab the existing entry.  if it doesn't exist, returns a list
    li = Data.setdefault((v["State"],v["City"],v["Area"]),[])
    li.append(v["Id"])

print "Data:%s" % (Data)

output:

Data:{('CA', 'San Francisco', 'North'): ['customer1', 'customer2'], ('CA', 'San Francisco', 'Embarcadero'): ['customer1']}

You're not limited to a very simple Id value, you can add pretty much anything you want to the list. If you expect to be doing this at a number of spots,look into https://docs.python.org/2/library/collections.html#collections.defaultdict, which kinda does a built-in setdefault.

In fact, you could add IDs to a dictionary instead of a list, it's all the same.

How to create nested dictionaries with duplicate keys in python

4 Answers4

Linked