How to iterate over nested dictionary structure and operate on each string?

Question

I have a dictionary that may contain any arbitrary order of strings, lists of strings or nested dictionaries that ultimate terminate in strings. I would like to iterate over this dictionary and perform an action on each string.

This question is close to what I'm looking for but I was unsuccessful in applying that solution to my own.

I need to apply the function os.path.expanduser() to each string in the following dictionary:

x = dict(
    dir = dict(
        wd = '~/Desktop/WD',
        pymodule = [
            '~/Documents/PythonModule',
            '/Users/Username/Documents/PythonModule2'
        ],
        album = '~/Desktop/Album'
    ),
    file = dict(
        XML = '~/Downloads/data.xml',
        CSV = '~/Downloads/data.csv'
    )
)

Ideally I would like to define a class that when called on a normal dictionary, will apply os.path.expanduser() on each string element of that dictionary.

class MyDict:
    def __init__(d):
        self.d = d
        # some operation to apply os.path.expanduser() on each string element of 'd'

How can I achieve this?

Aykhan Hagverdili · Accepted Answer · 2018-11-13T08:35:58.530

This can be easily done with a recursive function. Let's have a look at an example implementation. Here we will map all the strings in the given container to a given function, we will also use List Comprehensions and Dictionary Comprehensions to mimic the original nested structure. In addition, isinstance built in function is used to check the type of a given parameter:

def convert(func, d):
  if (isinstance(d, str)):
    return func(d)
  elif (isinstance(d, dict)):
    return {key : convert(func, d[key]) for key in d}
  elif (isinstance(d, list)):
    return [convert(func, val) for val in d]

Applies func to every string in the container. Test it with your example dictionary and os.path.expanduser:

x = dict(
    dir = dict(
        wd = '~/Desktop/WD',
        pymodule = [
            '~/Documents/PythonModule',
            '/Users/Username/Documents/PythonModule2'
        ],
        album = '~/Desktop/Album'
    ),
    file = dict(
        XML = '~/Downloads/data.xml',
        CSV = '~/Downloads/data.csv'
    )
)


import os
x = convert(os.path.expanduser, x)
print(x)

And sure enough you get the desired output:

{'dir': {'wd': '/home/runner/Desktop/WD', 'pymodule': ['/home/runner/Documents/PythonModule', '/Users/Username/Documents/PythonModule2'], 'album': '/home/runner/Desktop/Album'}, 'file': {'XML': '/home/runner/Downloads/data.xml', 'CSV': '/home/runner/Downloads/data.csv'}}

Great answer, and extra thank you for generalizing the `convert()` function to accept any input function, not just `os.path.expanduser()` — tsouchlarakis, Nov 12 '18 at 22:32

Henning Koehler · Answer 2 · 2018-11-12T08:27:30.160

2

Here's a function that takes a nested structure x as input, and return a similarly nested structure where all strings have been expanded:

def expand(x):
    if isinstance(x, str):
        return os.path.expanduser(x)
    if isinstance(x, dict):
        return { key : expand(x[key]) for key in x }
    if isinstance(x, list):
        return [ expand(elem) for elem in x ]
    return x

So e.g. calling it with

expand({1: '~/Hello', 2: ['~/World', '~/foo']})

will return

{1: '/home/hkoehler/Hello', 2: ['/home/hkoehler/World', '/home/hkoehler/foo']}

edited Nov 12 '18 at 08:27

answered Nov 12 '18 at 07:54

Henning Koehler

2,456
1
16
20

While this might answer the authors question, it lacks some explaining words and/or links to documentation. Raw code snippets are not very helpful without some phrases around them. You may also find [how to write a good answer](https://stackoverflow.com/help/how-to-answer) very helpful. Please edit your answer. – hellow Nov 12 '18 at 08:10
This is a fantastic answer that addresses all of my concerns in the original question. It works on a dictionary of arbitrary depth and that was exactly my idea. Thanks! – tsouchlarakis Nov 12 '18 at 22:32

score 0 · Answer 3 · answered Nov 12 '18 at 08:31

Here's a function that will do it:

import json
import os

x = dict(
    dir = dict(
        wd = '~/Desktop/WD',
        pymodule = [
            '~/Documents/PythonModule',
            '/Users/Username/Documents/PythonModule2'
        ],
        album = '~/Desktop/Album'
    ),
    file = dict(
        XML = '~/Downloads/data.xml',
        CSV = '~/Downloads/data.csv'
    )
)

def func(d):
    for key, value in d.items():
        if isinstance(value, dict):
            func(value)
        elif isinstance(value, str):
            d[key] = os.path.expanduser(value)
        elif isinstance(value, list):
            for i, element in enumerate(value):
                if isinstance(element, str):
                    value[i] = os.path.expanduser(element)

func(x)
print(json.dumps(x, indent=4))

Output:

{
    "dir": {
        "wd": "C:\\Users\\martineau/Desktop/WD",
        "pymodule": [
            "C:\\Users\\martineau/Documents/PythonModule",
            "/Users/Username/Documents/PythonModule2"
        ],
        "album": "C:\\Users\\martineau/Desktop/Album"
    },
    "file": {
        "XML": "C:\\Users\\martineau/Downloads/data.xml",
        "CSV": "C:\\Users\\martineau/Downloads/data.csv"
    }
}

That won't replace strings nested deep within lists. E.g. `[ [ '~/foo' ] ]`. — Henning Koehler, Nov 12 '18 at 08:33
@HenningKoehler: So? There's nothing like that in the OP's data structure. — martineau, Nov 12 '18 at 08:51

How to iterate over nested dictionary structure and operate on each string?

3 Answers3