grouping elements of a flattened dictionary into a sub list

Question

I have a flattened json (flattened using flatten json library). I have to now group them

{   'persons_0_address_building': '710',
    'persons_0_address_coord': '[123, 40]',
    'persons_0_address_street': 'Avenue 1',
    'persons_0_address_zipcode': '12345',
    'persons_0_cuisine': 'Chinese',
    'persons_0_grades_0_date': '2014-03-03T00:00:00.000Z',
    'persons_0_grades_0_grade': 'A',
    'persons_0_grades_0_score_x': 1,
    'persons_0_grades_0_score_y': 2,
    'persons_0_grades_1_date': '2011-11-23T00:00:00.000Z',
    'persons_0_grades_1_grade': 'A',
    'persons_0_grades_1_score_x': 11,
    'persons_0_grades_1_score_y': 22,
    'persons_0_id': '4d322fa8f552',
    'persons_0_name': 'Shash'

}

The desired result is as follows.

person_address = 
[
   {
    'building': '710',
    'coord': '[123, 40]',
    'street': 'Avenue 1',
    'zipcode': '12345',
    'id': '4d322fa8f552'
   }
]
person = 

[{
    'cuisine': 'Chinese',
    'id': '4d322fa8f552',
    'name': 'Shash'

}]

The basic idea is after grouping I can load each of the lists into a flat file.

What all I have tried so far:

Taking each of the key and checking for _, sorting them, and taking the result. This approach had lots of issues.
Tried using defaultdict but i am not able to even remotely get to this.

Is there any better way to get to this.

I _think_ you want to partially un-flatten this flattened data. Are you only extracting data for a single person, or is there also data for `persons_1`, `persons_2`, etc in your real data? — PM 2Ring, Jul 14 '18 at 11:42
@PM2Ring yes you are right. Sorry for the ambiquity from the earlier comment. I am extracting for multiple persons. — Shash, Jul 14 '18 at 13:21
@PM2Ring : How to partially unflatten the dict for each level, getting keys for each level — Shash, Jul 14 '18 at 13:23
It would be really helpful if you could show us some more realistic input that includes multiple people, and the expected output matching that input. The data doesn't need to include all the keys, just enough to unambiguously convey the correct structure. And then anyone who writes code for your question will be able to test their code. — PM 2Ring, Jul 14 '18 at 13:34
@PM2Ring : the exact input is what mehrdad-pedramfar has got as on output. I started with the json file, flattened it and got the above json. Now the idea is the merge individual key-value pairs traversing through `_` — Shash, Jul 15 '18 at 05:31

score 1 · Answer 1 · answered Jul 14 '18 at 14:40

I think i just found what you are looking for.
First you should define these two functions:

from functools import reduce
import operator


def get_from_dict(data_dict, map_list):
    return reduce(operator.getitem, map_list, data_dict)

def set_in_dict(data_dict, map_list, value):
    get_from_dict(data_dict, map_list[:-1])[map_list[-1]] = value

these two functions will set and get an item from a list using list of keys, visit here to learn more about these two.
then execute the code below to get your desired output:

final_dict = {}

for key, value in input_.items():
    keys_array = []
    for i in key.split('_'):
        try:
            index = int(i)
        except ValueError:
            keys_array.append(i)
            try:
                get_from_dict(final_dict, keys_array)
            except KeyError:
                set_in_dict(final_dict, keys_array, {})
            except IndexError:
                set_in_dict(final_dict, keys_array[:-1], {})
        else:
            keys_array.append(index)
            try:
                get_from_dict(final_dict, keys_array)
            except IndexError:
                if len(get_from_dict(final_dict, keys_array[:-1])) > 0:
                    get_from_dict(final_dict, keys_array[:-1]).append({})
                else:
                    set_in_dict(final_dict, keys_array, [])
            except KeyError:
                set_in_dict(final_dict, keys_array[:-1], [{}])
    else:
        set_in_dict(final_dict, keys_array, value)

assume that your input is in input_ and you desired output will be in final_dict.

the output will looks like this on your example:

{
  'persons': [
    {
      'address': 
        {
          'building': '710',
          'coord': '[123, 40]',
          'street': 'Avenue 1',
          'zipcode': '12345'
        },
      'cuisine': 'Chinese',
      'grades': [
        {
          'date': '2014-03-03T00:00:00.000Z',
          'grade': 'A',
          'score': {
            'x': 1,
            'y': 2
          }
        },
        {
          'date': '2011-11-23T00:00:00.000Z',
          'grade': 'A',
          'score': {
            'x': 11,
            'y': 22
          }
        }
      ],
     'id': '4d322fa8f552', 
     'name': 'Shash'
   }
  ]
}

this is the exact nested dictionary I started with to flatten. Now I need to merge values of the flattened dictionary to get to the desired result. The desired result is mentioned above — Shash, Jul 15 '18 at 05:27
@Shash "The desired result is mentioned above". Your description is too vague. You need to **show us** the expected output. — PM 2Ring, Jul 15 '18 at 05:42

grouping elements of a flattened dictionary into a sub list

1 Answers1