283

I have a nested dictionary. Is there only one way to get values out safely?

try:
    example_dict['key1']['key2']
except KeyError:
    pass

Or maybe python has a method like get() for nested dictionary ?

martineau
  • 119,623
  • 25
  • 170
  • 301
Arti
  • 7,356
  • 12
  • 57
  • 122
  • 1
    **See also:** https://stackoverflow.com/questions/14692690/access-nested-dictionary-items-via-a-list-of-keys – dreftymac Jan 18 '19 at 03:36
  • 1
    The code in your question is, in my view, already the best way to get nested values out of the dictionary. You can always specify a default value in the `except keyerror:` clause. – Peter Schorn Feb 16 '20 at 07:40

33 Answers33

545

You could use get twice:

example_dict.get('key1', {}).get('key2')

This will return None if either key1 or key2 does not exist.

Note that this could still raise an AttributeError if example_dict['key1'] exists but is not a dict (or a dict-like object with a get method). The try..except code you posted would raise a TypeError instead if example_dict['key1'] is unsubscriptable.

Another difference is that the try...except short-circuits immediately after the first missing key. The chain of get calls does not.


If you wish to preserve the syntax, example_dict['key1']['key2'] but do not want it to ever raise KeyErrors, then you could use the Hasher recipe:

class Hasher(dict):
    # https://stackoverflow.com/a/3405143/190597
    def __missing__(self, key):
        value = self[key] = type(self)()
        return value

example_dict = Hasher()
print(example_dict['key1'])
# {}
print(example_dict['key1']['key2'])
# {}
print(type(example_dict['key1']['key2']))
# <class '__main__.Hasher'>

Note that this returns an empty Hasher when a key is missing.

Since Hasher is a subclass of dict you can use a Hasher in much the same way you could use a dict. All the same methods and syntax is available, Hashers just treat missing keys differently.

You can convert a regular dict into a Hasher like this:

hasher = Hasher(example_dict)

and convert a Hasher to a regular dict just as easily:

regular_dict = dict(hasher)

Another alternative is to hide the ugliness in a helper function:

def safeget(dct, *keys):
    for key in keys:
        try:
            dct = dct[key]
        except KeyError:
            return None
    return dct

So the rest of your code can stay relatively readable:

safeget(example_dict, 'key1', 'key2')
Community
  • 1
  • 1
unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • 95
    so, python does not have beautiful solution for this case ?:( – Arti Sep 14 '14 at 13:29
  • I ran into a problem with a similar implementation. If you have d = {key1: None}, the first get will return None and then you'll have an exception ): I'm trying to figure it out a solution for this – Erico Jan 31 '18 at 19:30
  • 3
    The `safeget` method is in a lot of ways not very safe since it overwrites the original dictionary, meaning you can't safely do things like `safeget(dct, 'a', 'b') or safeget(dct, 'a')`. – neverfox Apr 11 '18 at 15:45
  • `safeget` never overwrites the original dictionary. It will either return the original dictionary, a value from the original dictionary, or `None`. – unutbu Apr 12 '18 at 14:39
  • @unutbu could you explain why overwriting the `dct` value in the for loop does not cause the `dct` argument (which is a mutable dictionary) to be overwritten? It works but I can't understand why. – Kurt Bourbaki Jun 08 '18 at 13:37
  • 8
    @KurtBourbaki: `dct = dct[key]` *reassigns* a new value to the *local variable* `dct`. This doesn't mutate the original dict (so the original dict is unaffected by `safeget`.) If, on the other hand, `dct[key] = ...` had been used, then the original dict would have been modified. In other words, in Python [names are bound to values](https://nedbatchelder.com/text/names.html). Assignment of a new value to a name does not affect the old value (unless there are no more references to the old value, in which case (in CPython) it will get garbage collected.) – unutbu Jun 08 '18 at 15:02
  • What about taking a copy of the dict in the beginning of method safeget? (using copy.deepcopy) – Martin Alexandersson Aug 10 '18 at 06:34
  • So far good unless `{ 'key1': [ { 'key2' : 'catchme' }, ] }` you try these out – ChandraKumar Apr 11 '20 at 10:16
  • 3
    The `safeget` method will also fail in case the key of a nested dict exists, but the value is null. It will throw `TypeError: 'NoneType' object is not subscriptable` in the next iteration – Stanley F. Jul 02 '20 at 07:23
  • I like the safeget method! In the spirit that a method should only return one data type how about returning an empty dictionary (`{}`) instead of None? – mjkrause Mar 26 '21 at 20:25
  • 4
    Starting with Python 3.4 you may use `with suppress(KeyError):`. See this answer: https://stackoverflow.com/a/45874251/1189659 – IODEV Apr 14 '21 at 17:09
84

By combining all of these answer here and small changes that I made, I think this function would be useful. its safe, quick, easily maintainable.

def deep_get(dictionary, keys, default=None):
    return reduce(lambda d, key: d.get(key, default) if isinstance(d, dict) else default, keys.split("."), dictionary)

Example :

from functools import reduce
def deep_get(dictionary, keys, default=None):
    return reduce(lambda d, key: d.get(key, default) if isinstance(d, dict) else default, keys.split("."), dictionary)

person = {'person':{'name':{'first':'John'}}}
print(deep_get(person, "person.name.first"))    # John

print(deep_get(person, "person.name.lastname")) # None

print(deep_get(person, "person.name.lastname", default="No lastname"))  # No lastname
Yuda Prawira
  • 12,075
  • 10
  • 46
  • 54
  • 1
    Perfect for Jinja2 templates – Thomas Mar 09 '18 at 12:30
  • This is a good solution though there also is a disadvantage: even if the first key is not available, or the value passed as the dictionary argument to the function is not a dictionary, the function will go from first element to the last one. Basically, it does this in all cases. – Arseny Jan 28 '19 at 12:04
  • 1
    `deep_get({'a': 1}, "a.b")` gives `None` but I would expect an exception like `KeyError` or something else. – Saddle Point Feb 27 '19 at 08:37
  • 3
    @edityouprofile. then you just need to do small modify to change return value from `None` to `Raise KeyError` – Yuda Prawira Aug 15 '19 at 20:41
  • Just FYI, doesn't handle nested lists. I.e. this won't work `deep_get(person, 'lastNames.0.name')` – blisstdev Jun 22 '23 at 22:55
81

You could also use python reduce:

def deep_get(dictionary, *keys):
    return reduce(lambda d, key: d.get(key) if d else None, keys, dictionary)
Yoav T
  • 811
  • 6
  • 3
  • 11
    Just wanted to mention that functools is [no longer a builtin in Python3](https://www.artima.com/weblogs/viewpost.jsp?thread=98196) and needs to be imported from functools, which makes this approach slightly less elegant. – yoniLavi Apr 12 '18 at 00:06
  • 11
    Slight correction to this comment: _reduce_ is no longer a built-in in Py3. But I don't see why this makes this any less elegant. It _does_ make it less suitable for a one-liner, but being a one-liner does not automatically qualify or disqualify something as being "elegant". – PaulMcG Jan 24 '20 at 13:36
  • 1
    Note that using try/except is usually better since it does not involve additional computational burden for checking the validity of the key. So if most of the time the keys exist, then I would recommend try/except paradigm for efficiency. – milembar Nov 12 '20 at 13:53
30

You can .get an empty dictionary, in the first stage.

example_dict.get('key1',{}).get('key2')
oli5679
  • 1,709
  • 1
  • 22
  • 34
18

Building up on Yoav's answer, an even safer approach:

def deep_get(dictionary, *keys):
    return reduce(lambda d, key: d.get(key, None) if isinstance(d, dict) else None, keys, dictionary)
Jose Alban
  • 7,286
  • 2
  • 34
  • 19
17

A recursive solution. It's not the most efficient but I find it a bit more readable than the other examples and it doesn't rely on functools.

def deep_get(d, keys):
    if not keys or d is None:
        return d
    return deep_get(d.get(keys[0]), keys[1:])

Example

d = {'meta': {'status': 'OK', 'status_code': 200}}
deep_get(d, ['meta', 'status_code'])     # => 200
deep_get(d, ['garbage', 'status_code'])  # => None

A more polished version

def deep_get(d, keys, default=None):
    """
    Example:
        d = {'meta': {'status': 'OK', 'status_code': 200}}
        deep_get(d, ['meta', 'status_code'])          # => 200
        deep_get(d, ['garbage', 'status_code'])       # => None
        deep_get(d, ['meta', 'garbage'], default='-') # => '-'
    """
    assert type(keys) is list
    if d is None:
        return default
    if not keys:
        return d
    return deep_get(d.get(keys[0]), keys[1:], default)
Pithikos
  • 18,827
  • 15
  • 113
  • 136
16

I suggest you to try python-benedict.

It is a dict subclass that provides keypath support and much more.

Installation: pip install python-benedict

from benedict import benedict

example_dict = benedict(example_dict, keypath_separator='.')

now you can access nested values using keypath:

val = example_dict['key1.key2']

# using 'get' method to avoid a possible KeyError:
val = example_dict.get('key1.key2')

or access nested values using keys list:

val = example_dict['key1', 'key2']

# using get to avoid a possible KeyError:
val = example_dict.get(['key1', 'key2'])

It is well tested and open-source on GitHub:

https://github.com/fabiocaccamo/python-benedict

Note: I am the author of this project

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Fabio Caccamo
  • 1,871
  • 19
  • 21
  • @perfecto25 thank you! I will release new features soon, stay tuned – Fabio Caccamo Jan 18 '20 at 15:31
  • @perfecto25 I added support to list indexes, eg. `d.get('a.b[0].c[-1]')` – Fabio Caccamo Jan 31 '20 at 12:05
  • The from_toml function doesn't seem to be implemented. And it can be difficult to import BeneDict. – DLyons Jul 30 '20 at 14:48
  • @DLyons you are wrong, in any case feel free to open an issue on GitHub. – Fabio Caccamo Jul 30 '20 at 16:17
  • 1
    Yes, it's there all right. Pity I missed it - would have save me some time. Benedict seems to have some very useful functionality. – DLyons Jul 31 '20 at 05:59
  • @DLyons thank you, if you have any idea for new features just let me know on GitHub. – Fabio Caccamo Jul 31 '20 at 11:04
  • Benedict seemed to install OK with poetry. But any variants of "from benedict import benedict" gave me cannot import BeneDict error. I tried a few things like deleting everything from __init__.py but in the end it seemed quicker to load and parse my small, regular toml file otherwise. – DLyons Jul 31 '20 at 11:15
  • Just open an issue here: https://github.com/fabiocaccamo/python-benedict/issues – Fabio Caccamo Jul 31 '20 at 13:30
10

While the reduce approach is neat and short, I think a simple loop is easier to grok. I've also included a default parameter.

def deep_get(_dict, keys, default=None):
    for key in keys:
        if isinstance(_dict, dict):
            _dict = _dict.get(key, default)
        else:
            return default
    return _dict

As an exercise to understand how the reduce one-liner worked, I did the following. But ultimately the loop approach seems more intuitive to me.

def deep_get(_dict, keys, default=None):

    def _reducer(d, key):
        if isinstance(d, dict):
            return d.get(key, default)
        return default

    return reduce(_reducer, keys, _dict)

Usage

nested = {'a': {'b': {'c': 42}}}

print deep_get(nested, ['a', 'b'])
print deep_get(nested, ['a', 'b', 'z', 'z'], default='missing')
zzz
  • 2,515
  • 4
  • 28
  • 38
  • I like the loop because it is way more flexible. For example, it is possible to use it to apply some lambda on the nested field, rather than getting it. – milembar Nov 12 '20 at 14:25
10

You can use pydash:

import pydash as _  #NOTE require `pip install pydash`

_.get(example_dict, 'key1.key2', default='Default')

https://pydash.readthedocs.io/en/latest/api.html

Nam G VU
  • 33,193
  • 69
  • 233
  • 372
steamdragon
  • 1,052
  • 13
  • 15
8

glom is a nice library that can into dotted queries too:

In [1]: from glom import glom

In [2]: data = {'a': {'b': {'c': 'd'}}}

In [3]: glom(data, "a.b.c")
Out[3]: 'd'

A query failure has a nice stack trace, indicating the exact failure spot:

In [4]: glom(data, "a.b.foo")
---------------------------------------------------------------------------
PathAccessError                           Traceback (most recent call last)
<ipython-input-4-2a3467493ac4> in <module>
----> 1 glom(data, "a.b.foo")

~/.cache/pypoetry/virtualenvs/neural-knapsack-dE7ihQtM-py3.8/lib/python3.8/site-packages/glom/core.py in glom(target, spec, **kwargs)
   2179 
   2180     if err:
-> 2181         raise err
   2182     return ret
   2183 

PathAccessError: error raised while processing, details below.
 Target-spec trace (most recent last):
 - Target: {'a': {'b': {'c': 'd'}}}
 - Spec: 'a.b.foo'
glom.core.PathAccessError: could not access 'foo', part 2 of Path('a', 'b', 'foo'), got error: KeyError('foo')

Safeguard with default:

In [5]: glom(data, "a.b.foo", default="spam")
Out[5]: 'spam'

The beauty of glom is in the versatile spec parameter. For example, one can easily extract all first names from the following data:

In [8]: data = {
   ...:     "people": [
   ...:         {"first_name": "Alice", "last_name": "Adams"},
   ...:         {"first_name": "Bob", "last_name": "Barker"}
   ...:     ]
   ...: }

In [9]: glom(data, ("people", ["first_name"]))
Out[9]: ['Alice', 'Bob']

Read the glom docs for more examples.

hoefling
  • 59,418
  • 12
  • 147
  • 194
5

Starting with Python 3.4 you may use with suppress (KeyError) to access nested json objects without worrying of Keyerror

from contextlib import suppress

with suppress(KeyError):
    a1 = json_obj['key1']['key2']['key3']
    a2 = json_obj['key4']['key5']['key6']
    a3 = json_obj['key7']['key8']['key9']

Courtesy of Techdragon. Have a look at his answer for further details: https://stackoverflow.com/a/45874251/1189659

IODEV
  • 1,706
  • 2
  • 17
  • 20
  • 1
    Note that if `key1` is missing, `a1` variable will not be set at all, leading to `NameError` when you attempt to use it – kolypto Jul 19 '21 at 14:48
4

for a second level key retrieving, you can do this:

key2_value = (example_dict.get('key1') or {}).get('key2')
Jacob CUI
  • 1,327
  • 15
  • 11
4

A simple class that can wrap a dict, and retrieve based on a key:

class FindKey(dict):
    def get(self, path, default=None):
        keys = path.split(".")
        val = None

        for key in keys:
            if val:
                if isinstance(val, list):
                    val = [v.get(key, default) if v else None for v in val]
                else:
                    val = val.get(key, default)
            else:
                val = dict.get(self, key, default)

            if not val:
                break

        return val

For example:

person = {'person':{'name':{'first':'John'}}}
FindDict(person).get('person.name.first') # == 'John'

If the key doesn't exist, it returns None by default. You can override that using a default= key in the FindDict wrapper -- for example`:

FindDict(person, default='').get('person.name.last') # == doesn't exist, so ''
Lee Benson
  • 11,185
  • 6
  • 43
  • 57
3

I adapted GenesRus and unutbu's answer in this very simple:

class new_dict(dict):
    def deep_get(self, *args, default=None):
        _empty_dict = {}
        out = self
        for key in args:
            out = out.get(key, _empty_dict)
        return out if out else default

it works with:

d = new_dict(some_data)
d.deep_get("key1", "key2", "key3", ..., default=some_value)
n4321d
  • 1,059
  • 2
  • 12
  • 31
2

After seeing this for deeply getting attributes, I made the following to safely get nested dict values using dot notation. This works for me because my dicts are deserialized MongoDB objects, so I know the key names don't contain .s. Also, in my context, I can specify a falsy fallback value (None) that I don't have in my data, so I can avoid the try/except pattern when calling the function.

from functools import reduce # Python 3
def deepgetitem(obj, item, fallback=None):
    """Steps through an item chain to get the ultimate value.

    If ultimate value or path to value does not exist, does not raise
    an exception and instead returns `fallback`.

    >>> d = {'snl_final': {'about': {'_icsd': {'icsd_id': 1}}}}
    >>> deepgetitem(d, 'snl_final.about._icsd.icsd_id')
    1
    >>> deepgetitem(d, 'snl_final.about._sandbox.sbx_id')
    >>>
    """
    def getitem(obj, name):
        try:
            return obj[name]
        except (KeyError, TypeError):
            return fallback
    return reduce(getitem, item.split('.'), obj)
Community
  • 1
  • 1
Donny Winston
  • 2,324
  • 1
  • 15
  • 13
  • 8
    `fallback` is not actually used in the function. – 153957 Sep 27 '16 at 13:25
  • Note that this does not work for keys that contain a `.` – JW. Jul 07 '17 at 22:11
  • When we call obj[name] why not obj.get(name, fallback) and avoid the try-catch (if you do want the try-catch, then return fallback, not None) – Jesper - jtk.eth Dec 15 '17 at 20:59
  • Thanks @153957. I fixed it. And yes @JW, this works for my use case. You could add a `sep=','` keyword arg to generalize for given (sep, fallback) conditions. And @denvar, if `obj` is say of type `int` after a sequence of the reduce, then obj[name] raises a TypeError, which I catch. If I used obj.get(name) or obj.get(name, fallback) instead, it would raise an AttributeError, so either way I'd need to catch. – Donny Winston Dec 17 '17 at 07:50
1

An adaptation of unutbu's answer that I found useful in my own code:

example_dict.setdefaut('key1', {}).get('key2')

It generates a dictionary entry for key1 if it does not have that key already so that you avoid the KeyError. If you want to end up a nested dictionary that includes that key pairing anyway like I did, this seems like the easiest solution.

GenesRus
  • 1,057
  • 6
  • 16
1

Yet another function for the same thing, also returns a boolean to represent whether the key was found or not and handles some unexpected errors.

'''
json : json to extract value from if exists
path : details.detail.first_name
            empty path represents root

returns a tuple (boolean, object)
        boolean : True if path exists, otherwise False
        object : the object if path exists otherwise None

'''
def get_json_value_at_path(json, path=None, default=None):

    if not bool(path):
        return True, json
    if type(json) is not dict :
        raise ValueError(f'json={json}, path={path} not supported, json must be a dict')
    if type(path) is not str and type(path) is not list:
        raise ValueError(f'path format {path} not supported, path can be a list of strings like [x,y,z] or a string like x.y.z')

    if type(path) is str:
        path = path.strip('.').split('.')
    key = path[0]
    if key in json.keys():
        return get_json_value_at_path(json[key], path[1:], default)
    else:
        return False, default

example usage:

my_json = {'details' : {'first_name' : 'holla', 'last_name' : 'holla'}}
print(get_json_value_at_path(my_json, 'details.first_name', ''))
print(get_json_value_at_path(my_json, 'details.phone', ''))

(True, 'holla')

(False, '')

penduDev
  • 4,743
  • 35
  • 37
1

There are already lots of good answers but I have come up with a function called get similar to lodash get in JavaScript land that also supports reaching into lists by index:

def get(value, keys, default_value = None):
'''
    Useful for reaching into nested JSON like data
    Inspired by JavaScript lodash get and Clojure get-in etc.
'''
  if value is None or keys is None:
      return None
  path = keys.split('.') if isinstance(keys, str) else keys
  result = value
  def valid_index(key):
      return re.match('^([1-9][0-9]*|[0-9])$', key) and int(key) >= 0
  def is_dict_like(v):
      return hasattr(v, '__getitem__') and hasattr(v, '__contains__')
  for key in path:
      if isinstance(result, list) and valid_index(key) and int(key) < len(result):
          result = result[int(key)] if int(key) < len(result) else None
      elif is_dict_like(result) and key in result:
          result = result[key]
      else:
          result = default_value
          break
  return result

def test_get():
  assert get(None, ['foo']) == None
  assert get({'foo': 1}, None) == None
  assert get(None, None) == None
  assert get({'foo': 1}, []) == {'foo': 1}
  assert get({'foo': 1}, ['foo']) == 1
  assert get({'foo': 1}, ['bar']) == None
  assert get({'foo': 1}, ['bar'], 'the default') == 'the default'
  assert get({'foo': {'bar': 'hello'}}, ['foo', 'bar']) == 'hello'
  assert get({'foo': {'bar': 'hello'}}, 'foo.bar') == 'hello'
  assert get({'foo': [{'bar': 'hello'}]}, 'foo.0.bar') == 'hello'
  assert get({'foo': [{'bar': 'hello'}]}, 'foo.1') == None
  assert get({'foo': [{'bar': 'hello'}]}, 'foo.1.bar') == None
  assert get(['foo', 'bar'], '1') == 'bar'
  assert get(['foo', 'bar'], '2') == None
Peter Marklund
  • 1,033
  • 10
  • 9
1

Here is a solution based on the unutbu's function answer plus:

  1. python naming guidelines
  2. default value as a parameter
  3. not using try but just checking if key is on object
def safe_get(dictionary, *keys, default=None):
    for key in keys:
        if key not in dictionary:
            return default
        dictionary = dictionary[key]
    return dictionary
Yerson Lasso
  • 11
  • 1
  • 1
1

The simplest way to do it without using libraries or writing functions and for a small number quick uses, the simplest way to do it is to use the get(..., {}) pattern as many times as needed, for example:

example_dict.get('key1', {}).get('key2', {}).get('key3', {}).get('key4', {})
mher
  • 369
  • 3
  • 7
  • for one-time use it's simple, concise, and fairly readable. If this is a regular use, a function makes more sense – Sigmatics Jun 27 '23 at 07:55
0

Since raising an key error if one of keys is missing is a reasonable thing to do, we can even not check for it and get it as single as that:

def get_dict(d, kl):
  cur = d[kl[0]]
  return get_dict(cur, kl[1:]) if len(kl) > 1 else cur
Ben Usman
  • 7,969
  • 6
  • 46
  • 66
0

Little improvement to reduce approach to make it work with list. Also using data path as string divided by dots instead of array.

def deep_get(dictionary, path):
    keys = path.split('.')
    return reduce(lambda d, key: d[int(key)] if isinstance(d, list) else d.get(key) if d else None, keys, dictionary)
Mike Kor
  • 876
  • 5
  • 14
0

A solution I've used that is similar to the double get but with the additional ability to avoid a TypeError using if else logic:

    value = example_dict['key1']['key2'] if example_dict.get('key1') and example_dict['key1'].get('key2') else default_value

However, the more nested the dictionary the more cumbersome this becomes.

0

For nested dictionary/JSON lookups, you can use dictor

pip install dictor

dict object

{
    "characters": {
        "Lonestar": {
            "id": 55923,
            "role": "renegade",
            "items": [
                "space winnebago",
                "leather jacket"
            ]
        },
        "Barfolomew": {
            "id": 55924,
            "role": "mawg",
            "items": [
                "peanut butter jar",
                "waggy tail"
            ]
        },
        "Dark Helmet": {
            "id": 99999,
            "role": "Good is dumb",
            "items": [
                "Shwartz",
                "helmet"
            ]
        },
        "Skroob": {
            "id": 12345,
            "role": "Spaceballs CEO",
            "items": [
                "luggage"
            ]
        }
    }
}

to get Lonestar's items, simply provide a dot-separated path, ie

import json
from dictor import dictor

with open('test.json') as data: 
    data = json.load(data)

print dictor(data, 'characters.Lonestar.items')

>> [u'space winnebago', u'leather jacket']

you can provide fallback value in case the key isnt in path

theres tons more options you can do, like ignore letter casing and using other characters besides '.' as a path separator,

https://github.com/perfecto25/dictor

perfecto25
  • 772
  • 9
  • 13
0

I little changed this answer. I added checking if we're using list with numbers. So now we can use it whichever way. deep_get(allTemp, [0], {}) or deep_get(getMinimalTemp, [0, minimalTemperatureKey], 26) etc

def deep_get(_dict, keys, default=None):
    def _reducer(d, key):
        if isinstance(d, dict):
            return d.get(key, default)
        if isinstance(d, list):
            return d[key] if len(d) > 0 else default
        return default
    return reduce(_reducer, keys, _dict)
Darex1991
  • 855
  • 1
  • 10
  • 24
  • fails on this: test_dict = {"a":{"b":[{"c":"value"}]}} self.assertEqual(safeget(test_dict, ["a", "b", 1, "c"], None) – Igor L. Oct 29 '20 at 15:51
0

Recursive method (мб пригодится)

Example dict:

foo = [{'feature_name': 'Sample Creator > Contract Details > Elements of the page',
  'scenarios': [{'scenario_name': 'SC, CD, Elements of the page',
                 'scenario_status': 'failed',
                 'scenario_tags': None,
                 'steps': [{'duration': 0,
                            'name': 'I open application Stage and login by '
                                    'SPT_LOGIN and password SPT_PWD',
                            'status': 'untested'},
                           {'duration': 0,
                            'name': 'I open Sample Creator query page',
                            'status': 'untested'},
                           {'duration': 7.78166389465332,
                            'name': 'I open application Stage and login by '
                                    'SPT_LOGIN and password SPT_PWD',
                            'status': 'passed'},
                           {'duration': 3.985326051712036,
                            'name': 'I open Sample Creator query page',
                            'status': 'passed'},
                           {'duration': 2.9063704013824463,
                            'name': 'Enter value: '
                                    'X-2008-CON-007,X-2011-CON-016 in '
                                    'textarea: project_text_area sleep: 1',
                            'status': 'passed'},
                           {'duration': 4.4447715282440186,
                            'name': 'I press on GET DATA',
                            'status': 'passed'},
                           {'duration': 1.1209557056427002,
                            'name': 'Verify the top table on Contract Details',
                            'status': 'passed'},
                           {'duration': 3.8173601627349854,
                            'name': 'I export contract_details table by offset '
                                    'x:100, y:150',
                            'status': 'passed'},
                           {'duration': 1.032956600189209,
                            'name': 'Check data of '
                                    'sc__cd_elements_of_the_page_1 and skip '
                                    'cols None',
                            'status': 'passed'},
                           {'duration': 0.04593634605407715,
                            'name': "Verify 'Number of Substances' column "
                                    'values',
                            'status': 'passed'},
                           {'duration': 0.10199904441833496,
                            'name': 'Substance Sample Details bottom table '
                                    'columns',
                            'status': 'passed'},
                           {'duration': 0.0009999275207519531,
                            'name': 'Verify the Substance Sample Details '
                                    'bottom table',
                            'status': 'passed'},
                           {'duration': 3.8558616638183594,
                            'name': 'I export substance_sample_details table '
                                    'by offset x:100, y:150',
                            'status': 'passed'},
                           {'duration': 1.0329277515411377,
                            'name': 'Check data of '
                                    'sc__cd_elements_of_the_page_2 and skip '
                                    'cols None',
                            'status': 'passed'},
                           {'duration': 0.2879970073699951,
                            'name': 'Click on AG-13369',
                            'status': 'passed'},
                           {'duration': 3.800830364227295,
                            'name': 'I export substance_sample_details table '
                                    'by offset x:100, y:150',
                            'status': 'passed'},
                           {'duration': 1.0169551372528076,
                            'name': 'Check data of '
                                    'sc__cd_elements_of_the_page_3 and skip '
                                    'cols None',
                            'status': 'passed'},
                           {'duration': 1.7484464645385742,
                            'name': 'Select all cells, table: 2',
                            'status': 'passed'},
                           {'duration': 3.812828779220581,
                            'name': 'I export substance_sample_details table '
                                    'by offset x:100, y:150',
                            'status': 'passed'},
                           {'duration': 1.0029594898223877,
                            'name': 'Check data of '
                                    'sc__cd_elements_of_the_page_2 and skip '
                                    'cols None',
                            'status': 'passed'},
                           {'duration': 1.6729373931884766,
                            'name': 'Set window size x:800, y:600',
                            'status': 'passed'},
                           {'duration': 30.145705699920654,
                            'name': 'All scrollers are placed on top 6 and far '
                                    'left 8',
                            'status': 'failed'}]}]},
  {'feature_name': 'Sample Creator > Substance Sample History > Elements of the '
                  'page',
  'scenarios': [{'scenario_name': 'SC, SSH, Elements of the page',
                 'scenario_status': 'passed',
                 'scenario_tags': None,
                 'steps': [{'duration': 0,
                            'name': 'I open application Stage and login by '
                                    'SPT_LOGIN and password SPT_PWD',
                            'status': 'untested'},
                           {'duration': 0,
                            'name': 'I open Sample Creator query page',
                            'status': 'untested'},
                           {'duration': 7.305850505828857,
                            'name': 'I open application Stage and login by '
                                    'SPT_LOGIN and password SPT_PWD',
                            'status': 'passed'},
                           {'duration': 3.500955104827881,
                            'name': 'I open Sample Creator query page',
                            'status': 'passed'},
                           {'duration': 3.0419492721557617,
                            'name': 'Enter value: NOA401800 SYN-NOA '
                                    'A,S4A482070C SYN-ISN-OLD '
                                    'O,S04A482167T,S04A482190Y,CSAA796564,CSCD106701 '
                                    'in textarea: id_text_area sleep: 1',
                            'status': 'passed'},
                           {'duration': 49.567158460617065,
                            'name': 'I press on GET DATA',
                            'status': 'passed'},
                           {'duration': 0.13904356956481934,
                            'name': 'Open substance_sample_history',
                            'status': 'passed'},
                           {'duration': 1.1039845943450928,
                            'name': 'Columns displayed',
                            'status': 'passed'},
                           {'duration': 3.881945848464966,
                            'name': 'I export export_parent_table table by '
                                    'offset x:100, y:150',
                            'status': 'passed'},
                           {'duration': 1.0334820747375488,
                            'name': 'Check data of '
                                    'sc__ssh_elements_of_the_page_1 and skip '
                                    'cols None',
                            'status': 'passed'},
                           {'duration': 0.0319981575012207,
                            'name': "Title is 'Additional Details for Marked "
                                    "Rows'",
                            'status': 'passed'},
                           {'duration': 0.08897256851196289,
                            'name': 'Columns displayed (the same as in top '
                                    'table)',
                            'status': 'passed'},
                           {'duration': 25.192569971084595,
                            'name': 'Verify the content of the bottom table',
                            'status': 'passed'},
                           {'duration': 4.308935880661011,
                            'name': 'I export '
                                    'additional_details_for_marked_rows table '
                                    'by offset x:100, y:150',
                            'status': 'passed'},
                           {'duration': 1.0089836120605469,
                            'name': 'Check data of '
                                    'sc__ssh_elements_of_the_page_1 and skip '
                                    'cols None',
                            'status': 'passed'}]}]}]

Code:

def get_keys(_dict: dict, prefix: list):
    prefix += list(_dict.keys())
    return prefix


def _loop_elements(elems:list, prefix=None, limit=None):
    prefix = prefix or []
    limit = limit or 9
    try:
        if len(elems) != 0 and isinstance(elems, list):
            for _ in elems:
                if isinstance(_, dict):
                    get_keys(_, prefix)
                    for item in _.values():
                        _loop_elements(item, prefix, limit)
        return prefix[:limit]
    except TypeError:
        return


>>>goo = _loop_elements(foo,limit=9)
>>>goo
['feature_name', 'scenarios', 'scenario_name', 'scenario_status', 'scenario_tags', 'steps', 'duration', 'name', 'status']
Igor Z
  • 601
  • 6
  • 7
0
def safeget(_dct, *_keys):
    if not isinstance(_dct, dict): raise TypeError("Is not instance of dict")
    def foo(dct, *keys):
        if len(keys) == 0: return dct
        elif not isinstance(_dct, dict): return None
        else: return foo(dct.get(keys[0], None), *keys[1:])
    return foo(_dct, *_keys)

assert safeget(dict()) == dict()
assert safeget(dict(), "test") == None
assert safeget(dict([["a", 1],["b", 2]]),"a", "d") == None
assert safeget(dict([["a", 1],["b", 2]]),"a") == 1
assert safeget({"a":{"b":{"c": 2}},"d":1}, "a", "b")["c"] == 2
avelot
  • 1
  • 1
  • 3
0

I have written a package deepextract that does exactly what you want: https://github.com/ya332/deepextract You can do

from deepextract import deepextract
# Demo: deepextract.extract_key(obj, key)
deeply_nested_dict = {
    "items": {
        "item": {
            "id": {
                "type": {
                    "donut": {
                        "name": {
                            "batters": {
                                "my_target_key": "my_target_value"
                            }
                        }
                    }
                }
            }
        }
    }
}
print(deepextract.extract_key(deeply_nested_dict, "my_target_key") == "my_target_value")

returns

True
Yigit Alparslan
  • 1,377
  • 1
  • 8
  • 13
0

My implementation that descends into sub-dicts, ignores None values, but fails with a TypeError if anything else is discovered

def deep_get(d: dict, *keys, default=None):
    """ Safely get a nested value from a dict

    Example:
        config = {'device': None}
        deep_get(config, 'device', 'settings', 'light')
        # -> None
        
    Example:
        config = {'device': True}
        deep_get(config, 'device', 'settings', 'light')
        # -> TypeError

    Example:
        config = {'device': {'settings': {'light': 'bright'}}}
        deep_get(config, 'device', 'settings', 'light')
        # -> 'light'

    Note that it returns `default` is a key is missing or when it's None.
    It will raise a TypeError if a value is anything else but a dict or None.
    
    Args:
        d: The dict to descend into
        keys: A sequence of keys to follow
        default: Custom default value
    """
    # Descend while we can
    try:
        for k in keys:
            d = d[k]
    # If at any step a key is missing, return default
    except KeyError:
        return default
    # If at any step the value is not a dict...
    except TypeError:
        # ... if it's a None, return default. Assume it would be a dict.
        if d is None:
            return default
        # ... if it's something else, raise
        else:
            raise
    # If the value was found, return it
    else:
        return d
kolypto
  • 31,774
  • 17
  • 105
  • 99
0

If you want to use another library for a solution, this is work best

https://github.com/maztohir/dict-path

from dict-path import DictPath

data_dict = {
  "foo1": "bar1",
  "foo2": "bar2",
  "foo3": {
     "foo4": "bar4",
     "foo5": {
        "foo6": "bar6",
        "foo7": "bar7",
     },
  }
}

data_dict_path = DictPath(data_dict)
data_dict_path.get('key1/key2/key3')
Rohit Banga
  • 18,458
  • 31
  • 113
  • 191
maztohir
  • 9
  • 1
0

You could use a NestedDict from the open-source ndicts package (I am the author), which has a safe get method exactly like dictionaries.

>>> from ndicts import NestedDict
>>> nd = NestedDict({"key1": {"key2": 0}}
>>> nd.get(("key1", "key2))
0
>>> nd.get("asd")

edd313
  • 1,109
  • 7
  • 20
0

There is no need in external python module, a single function is enough. There is no need to join keys in one string, if argument passing by *args is available.

def key_chain(data, *args, default=None): 
    for key in args:
        if isinstance(data, dict):
            data = data.get(key, default)
        elif isinstance(data, (list, tuple)) and isinstance(key, int):
            try:
                data = data[key]
            except IndexError:
                return default
        else:
            return default
    return data

So just call

key_chain(example_dict, "key1", "key2")

It doesn't crash with TypeError on the dictonary example_dict = {'key1': 1} unlike the highest score answer. It supports integer keys for lists and tuples and default value if any key is missed.

More examples of usage https://gist.github.com/yaznahar/26bd3442467aff5d126d345cca0efcad

-2

You can use dotted:

pip install dotted

from dotted.collection import DottedDict

assert DottedDict(dict(foo=dict(bar="baz")))["foo"]["bar"] == "baz"
assert DottedDict(dict(foo=dict(bar="baz")))["foo.bar"] == "baz"
assert DottedDict(dict(foo=dict(bar="baz"))).get("lorem.ipsum", None) is None
assert DottedDict(dict(foo=dict(bar="baz"))).get("lorem.ipsum", "default") == "default"
BaiJiFeiLong
  • 3,716
  • 1
  • 30
  • 28
  • Unfortunately, this library is abandonware. Last release 7 years ago, and it no longer works with 3.10 after [`collections.abc`](https://docs.python.org/3/whatsnew/3.10.html#removed) was removed. – Nick K9 Nov 19 '22 at 17:12