flatten nested dict values in a list in Python 3

Question

I've got this data structure coming from Vimeo API

{'duration': 720, 
'language': 'sv',
'link': 'https://vimeo.com/neweuropefilmsale/incidentbyabank',
'name': 'INCIDENT BY A BANK',
'user': {
    'link': 'https://vimeo.com/neweuropefilmsales',
    'location': 'Warsaw, Poland',
    'name': 'New Europe Film Sales'
    }
}

I want to transofrm in

[720, "sv", "http..", "incident.." "http..", "Warsaw", "New Europe.."]

to load it in a Google spreadsheet. I also need to maintain consistence values order.

PS. I see similar questions but answers are not in Python 3

Thanks

Have you tried using the `csv` module with a header? The order won't matter if you use a header. — Markus Meskanen, Aug 08 '17 at 15:37
thanks for suggestion, but i want Google spreadsheet to update dinamically adding new rows every day. I don't think CSV is a solution — Gusepo, Aug 08 '17 at 15:43
What module are you using to update the google spreadsheets? Maybe you can download it to a local copy, edit that one with `csv`, and then upload back to google? — Markus Meskanen, Aug 08 '17 at 15:44
There's a `import_csv` function you can use. https://gspread.readthedocs.io/en/latest/#gspread.Client.import_csv — Markus Meskanen, Aug 08 '17 at 15:53

score 2 · Answer 1 · answered Aug 08 '17 at 15:52

I'm going to use the csv module to create a CSV file like you've described out of your data.

First, we should use a header row for your file, so the order doesn't matter, only dict keys do:

import csv

# This defines the order they'll show up in final file
fieldnames = [
    'name', 'link', 'duration', 'language',
    'user_name', 'user_link', 'user_location',
]

# Open the file with Python
with open('my_file.csv', 'w', newline='') as my_file:

    # Attach a CSV writer to the file with the desired fieldnames
    writer = csv.DictWriter(my_file, fieldnames)

    # Write the header row
    writer.writeheader()

Notice the DictWriter, this will allow us to write dicts based on their keys instead of the order (dicts are unordered pre-3.6). The above code will end up with a file like this:

name;link;duration;language;user_name;user_link;user_location

Which we can then add rows to, but let's convert your data first, so the keys match the above field names:

data = {
    'duration': 720, 
    'language': 'sv',
    'link': 'https://vimeo.com/neweuropefilmsale/incidentbyabank',
    'name': 'INCIDENT BY A BANK',
    'user': {
        'link': 'https://vimeo.com/neweuropefilmsales',
        'location': 'Warsaw, Poland',
        'name': 'New Europe Film Sales'
    }
}

for key, value in data['user'].items():
    data['user_{}'.format(key)] = value
del data['user']

This ends up with the data dictionary like this:

data = {
    'duration': 720, 
    'language': 'sv',
    'link': 'https://vimeo.com/neweuropefilmsale/incidentbyabank',
    'name': 'INCIDENT BY A BANK',
    'user_link': 'https://vimeo.com/neweuropefilmsales',
    'user_location': 'Warsaw, Poland',
    'user_name': 'New Europe Film Sales',
}

We can now simply insert this as a whole row to the CSV writer, and everything else is done automatically:

# Using the same writer from above, insert the data from above
writer.writerow(data)

That's it, now just import this into your Google spreadsheets :)

score 2 · Answer 2 · answered Aug 08 '17 at 15:55

This is a simple solution using recursion:

dictionary = {
    'duration': 720, 
    'language': 'sv',
    'link': 'https://vimeo.com/neweuropefilmsale/incidentbyabank',
    'name': 'INCIDENT BY A BANK',
    'user': {
        'link': 'https://vimeo.com/neweuropefilmsales',
        'location': 'Warsaw, Poland',
        'name': 'New Europe Film Sales'
    }
}


def flatten(current: dict, result: list=[]):
    if isinstance(current, dict):
        for key in current:
            flatten(current[key], result)
    else:
        result.append(current)

    return result


result = flatten(dictionary)
print(result)

Explanation: We call flatten() until we reach a value of the dictionary, that is not a dictionary itself (if isinstance(current, dict):). If we reach this value, we append it to our result list. It will work for any number of nested dictionaries.

See: How would I flatten a nested dictionary in Python 3? I used the same solution, but I've changed the result collection to be a list.

Thanks the flatten function at https://stackoverflow.com/questions/24448543/how-would-i-flatten-a-nested-dictionary-in-python-3?answertab=votes#tab-top is working also in a for loop, while yours add the previous elements for each iteration — Gusepo, Aug 09 '17 at 09:29
No, it's different to that. This method produces a list, and the one you linked to produces a dict. They are both almost identical recursive solutions. — tmck-code, Mar 29 '19 at 02:38

flatten nested dict values in a list in Python 3

2 Answers2