Merging a number of dictionaries maintaining duplicate values, but merging duplicate keys in Python

Question

I'm doing something wrong with my code for merging dictionaries.Here's what I got:

output1 = {'children': 
                    {'children':
                              {'children': 
                                        {'name': 'thriller.mp3', 'type': 'file'},
                               'name': 'pop',
                               'type': 'folder'},
                     'name': 'test_pendrive',
                     'type': 'folder'},
           'name': 'media',
           'type': 'folder'}
output2 = {'children': 
                    {'children':
                              {'children': 
                                        {'name': 'paranoid.mp3', 'type': 'file'},
                               'name': 'metal',
                               'type': 'folder'},
                     'name': 'test_pendrive',
                     'type': 'folder'},
           'name': 'media',
           'type': 'folder'}

My code is looking like this at the moment:

from collections import defaultdict

l1 = list(output2.items())
l2 = list(output.items())

l3 = l1+l2
d1 = defaultdict(list)
for k,v in l3:
   d1[k].append(v)

Which gives the output:

{
    "name": ["media", "media"],
    "type": ["folder", "folder"],
    "children": [{
            "name": "test_pendrive",
            "type": "folder",
            "children": {
                "name": "dance",
                "type": "folder",
                "children": {
                    "name": "billie_jean.mp3",
                    "type": "file"
                }
            }
        },
        {
            "name": "test_pendrive",
            "type": "folder",
            "children": {
                "name": "pop",
                "type": "folder",
                "children": {
                    "name": "thriller.mp3",
                    "type": "file"
                }
            }
        }
    ]
}

That is almost what I want. My final goal is this:

{
    "name": ["media", "media"],
    "type": ["folder", "folder"],
    "children": [{
            "name": "test_pendrive",
            "type": "folder",
            "children": [{
                "name": "dance",
                "type": "folder",
                "children": {
                    "name": "billie_jean.mp3",
                    "type": "file"
                }
            },{
                "name": "pop",
                "type": "folder",
                "children": {
                    "name": "thriller.mp3",
                    "type": "file"
                }
            }]
      }
}

Where am I going wrong? Is this the best method to achieve that goal? Thanks in advance.

I might be wrong, but with a little more context (where is the data coming from, what is your end goal), I think we might be able to help you better. It looks to me as if you are trying to programatically reproduce the effect of merging two file trees. — silel, May 05 '17 at 14:52
It's not really clear what you want to do. For example, in the original trees, "pop" is inside "test_pendrive", whereas afterwards it isn't. But "dance" does remain inside "test_pendrive". Why are they treated differently? Also, your code puts singletons into a list, whereas your output suggests you don't want to do that? — strubbly, May 05 '17 at 14:58
@strubbly Weel, pop remains inside test_pendrive. Now, pop and dance are in a list as children of test_pendrive. — Mikael, May 05 '17 at 15:05
@silel I'm doing precisely that. I posted a question, but people were not very helping, so I took my shot, and this is the furthest I could get. — Mikael, May 05 '17 at 15:07
Does the original question still exist? Can you give us a link? If not, can you explain how the data was produced and why you need to programatically merge file trees? (see [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem)) — silel, May 06 '17 at 20:14
@silel the link for the other question: http://stackoverflow.com/questions/43641813/merging-arbitrarily-nested-dictionaries-maintaining-keys-and-putting-results-w — Mikael, May 07 '17 at 00:58
@silel I'm merging file trees because I have a set of paths to files in the format `"/media/lol/123.txt"` and I want map the file tree to a dictionary,and make it to a JSON. — Mikael, May 07 '17 at 01:00
well then, maybe you should have a look at [this thread](http://stackoverflow.com/questions/25226208/represent-directory-tree-as-json) — silel, May 09 '17 at 00:04

strubbly · Answer 1 · 2017-05-06T06:59:32.467

Although you have given one clear example of what you want to achieve, that doesn't clearly specify your objective in general. For example, what result would you expect for this?

output1 = {'children': 
                {'children':
                      {'children': 
                               {'name': 'thriller.mp3', 'type': 'file'},
                           'name': 'pop',
                           'type': 'folder'},
                 'name': 'test_pendrive',
                 'type': 'folder'},
           'name': 'media1',
           'type': 'folder'}
output2 = {'children': 
                {'children':
                      {'children': 
                                {'name': 'paranoid.mp3', 'type': 'file'},
                           'name': 'metal',
                           'type': 'folder'},
                 'name': 'test_pendrive',
                 'type': 'folder'},
       'name': 'media2',
       'type': 'folder'}

which is your example but with the top names changed? Or what about this one?

output1 = {'children': 
                {'children':
                      {'children': 
                               {'name': 'thriller.mp3', 'type': 'file'},
                           'name': 'pop',
                           'type': 'folder'},
                 'name': 'test_pendrive',
                 'type': 'folder'},
           'name': 'media1',
           'type': 'folder'}
output2 = {'children': 
                {'children':
                      {'children': 
                                {'name': 'paranoid.mp3', 'type': 'file'},
                           'name': 'metal',
                           'type': 'folder'},
                 'type': 'folder'},
       'name': 'media2',
       'type': 'folder'}

which is your example, but with one name missing. Or this one?

output1 = {'children': 
                {'children':
                      {'children': {} },
           'a': 'b',
           'c': 'd'}
output2 = {'children': 
                {'children':
                      {'children': {"a" : "b"} },
           'a': 'b',
           'c': 'd'}

I've taken a guess at what you want to do, which is to merge the values in the dictionaries, but to recursively do this down the 'children' tree structure. This doesn't give the output you suggest but it might make sense.

from collections import defaultdict

def merge_children(*dicts):
    result = defaultdict(list)
    for d in dicts:
        for key in d:
            result[key].append(d[key])
    children = result["children"]
    if len(children)>1:
        result["children"] = merge_children(*children)
    return dict(result)

This gives the following:

{'children': 
    {'children': 
        {'children': 
            {'children': [],
             'name': ['thriller.mp3', 'paranoid.mp3'],
             'type': ['file', 'file']
            },
         'name': ['pop', 'metal'],
         'type': ['folder', 'folder']
        },
     'name': ['test_pendrive', 'test_pendrive'],
     'type': ['folder', 'folder']
    },
 'name': ['media', 'media'],
 'type': ['folder', 'folder']
}

Alternatively perhaps you want to treat the 'name' key specially as well. In which case you should perhaps be using a different data structure to represent your trees - something like this:

output1 = {
 'media' : {'type' : 'folder',
             'children' : {
      'test_pendrive' :
                {'type' : 'folder',
                 'children' : {
          'pop' :
                    {'type': 'folder',
                     'children' :  
                           {'thriller.mp3' : {'type': 'file'},}
                     }}}}}}
output2 = {
 'media' : {'type' : 'folder',
             'children' : {
      'test_pendrive' :
                {'type' : 'folder',
                 'children' : {
          'metal' :
                    {'type': 'folder',
                     'children' :  
                           {'paranoid.mp3' : {'type': 'file'},}
                     }}}}}}

If you use a structure like this then merging the dictionaries will indeed merge children with the same name because the name is the key against which the merging takes place. Which is not the case in your original version.

This not exactly what I was looking for, because paranoid is listed as children of pop, and it's not. — Mikael, May 05 '17 at 15:09
OK see what you mean. But in your example, in the top dictionary, the names are merged, whereas in the lower directories they are not. What would happen if, for example, the "name"s were not the same in the output1 and output2? — strubbly, May 05 '17 at 15:33

Merging a number of dictionaries maintaining duplicate values, but merging duplicate keys in Python

1 Answers1