Python: load variables in a dict into namespace

Question

I want to use a bunch of local variables defined in a function, outside of the function. So I am passing x=locals() in the return value.

How can I load all the variables defined in that dictionary into the namespace outside the function, so that instead of accessing the value using x['variable'], I could simply use variable.

Answer 1

Rather than create your own object, you can use argparse.Namespace:

from argparse import Namespace
ns = Namespace(**mydict)

To do the inverse:

mydict = vars(ns)

Answer 2

Consider the Bunch alternative:

class Bunch(object):
  def __init__(self, adict):
    self.__dict__.update(adict)

so if you have a dictionary d and want to access (read) its values with the syntax x.foo instead of the clumsier d['foo'], just do

x = Bunch(d)

this works both inside and outside functions -- and it's enormously cleaner and safer than injecting d into globals()! Remember the last line from the Zen of Python...:

>>> import this
The Zen of Python, by Tim Peters
   ...
Namespaces are one honking great idea -- let's do more of those!

Answer 3

This is perfectly valid case to import variables in one local space into another local space as long as one is aware of what he/she is doing. I have seen such code many times being used in useful ways. Just need to be careful not to pollute common global space.

You can do the following:

adict = { 'x' : 'I am x', 'y' : ' I am y' }
locals().update(adict)
blah(x)
blah(y)

Answer 4

Importing variables into a local namespace is a valid problem and often utilized in templating frameworks.

Return all local variables from a function:

return locals()

Then import as follows:

r = fce()
for key in r.keys():
   exec(key + " = r['" + key + "']")

Answer 5

The Bunch answer is ok but lacks recursion and proper __repr__ and __eq__ builtins to simulate what you can already do with a dict. Also the key to recursion is not only to recurse on dicts but also on lists, so that dicts inside lists are also converted.

These two options I hope will cover your needs (you might have to adjust the type checks in __elt() for more complex objects; these were tested mainly on json imports so very simple core types).

1. The Bunch approach (as per previous answer) - object takes a dict and converts it recursively. repr(obj) will return Bunch({...}) that can be re-interpreted into an equivalent object.

class Bunch(object):
    def __init__(self, adict):
        """Create a namespace object from a dict, recursively"""
        self.__dict__.update({k: self.__elt(v) for k, v in adict.items()})

    def __elt(self, elt):
        """Recurse into elt to create leaf namespace objects"""
        if type(elt) is dict:
            return type(self)(elt)
        if type(elt) in (list, tuple):
            return [self.__elt(i) for i in elt]
        return elt
    
    def __repr__(self):
        """Return repr(self)."""
        return "%s(%s)" % (type(self).__name__, repr(self.__dict__))

    def __eq__(self, other):
        if hasattr(other, '__dict__'):
            return self.__dict__ == other.__dict__
        return NotImplemented
        # Use this to allow comparing with dicts:
        #return self.__dict__ == (other.__dict__ if hasattr(other, '__dict__') else other)

2. The SimpleNamespace approach - since types.SimpleNamespace already implements __repr__ and __eq__, all you need is to implement a recursive __init__ method:

import types
class RecursiveNamespace(types.SimpleNamespace):
    # def __init__(self, /, **kwargs):  # better, but Python 3.8+
    def __init__(self, **kwargs):
        """Create a SimpleNamespace recursively"""
        self.__dict__.update({k: self.__elt(v) for k, v in kwargs.items()})
        
    def __elt(self, elt):
        """Recurse into elt to create leaf namespace objects"""
        if type(elt) is dict:
            return type(self)(**elt)
        if type(elt) in (list, tuple):
            return [self.__elt(i) for i in elt]
        return elt

    # Optional, allow comparison with dicts:
    #def __eq__(self, other):
    #    return self.__dict__ == (other.__dict__ if hasattr(other, '__dict__') else other)

The RecursiveNamespace class takes keyword arguments, which can of course come from a de-referenced dict (ex **mydict)

---

Now let's put them to the test (argparse.Namespace added for comparison, although it's nested dict is manually converted):

from argparse import Namespace
from itertools import combinations
adict = {'foo': 'bar', 'baz': [{'aaa': 'bbb', 'ccc': 'ddd'}]}
a = Bunch(adict)
b = RecursiveNamespace(**adict)
c = Namespace(**adict)
c.baz[0] = Namespace(**c.baz[0])
for n in ['a', 'b', 'c']:
    print(f'{n}:', str(globals()[n]))
for na, nb in combinations(['a', 'b', 'c'], 2):
    print(f'{na} == {nb}:', str(globals()[na] == globals()[nb]))

The result is:

a: Bunch({'foo': 'bar', 'baz': [Bunch({'aaa': 'bbb', 'ccc': 'ddd'})]})
b: RecursiveNamespace(foo='bar', baz=[RecursiveNamespace(aaa='bbb', ccc='ddd')])
c: Namespace(foo='bar', baz=[Namespace(aaa='bbb', ccc='ddd')])
a == b: True
a == c: True
b == c: False

Although those are different classes, because they both (a and b) have been initialized to equivalent namespaces and their __eq__ method compares the namespace only (self.__dict__), comparing two namespace objects returns True. For the case of comparing with argparse.Namespace, for some reason only Bunch works and I'm unsure why (please comment if you know, I haven't looked much further as types.SimpleNameSpace is a built-in implementation).

You might also notice that I recurse using type(self)(...) rather than using the class name - this has two advantages: first the class can be renamed without having to update recursive calls, and second if the class is subclassed we'll be recursing using the subclass name. It's also the name used in __repr__ (type(self).__name__).

EDIT 2021-11-27:

1. Modified the Bunch.__eq__ method to make it safe against type mismatch.

2. Added/modified optional __eq__ methods (commented out) to allow comparing with the original dict and argparse.Namespace(**dict) (note that the later is not recursive but would still be comparable with other classes as the sublevel structs would compare fine anyway).

Answer 6

Used following snippet (PY2) to make recursive namespace from my dict(yaml) configs:

class NameSpace(object):
    def __setattr__(self, key, value):
        raise AttributeError('Please don\'t modify config dict')


def dump_to_namespace(ns, d):
    for k, v in d.iteritems():
        if isinstance(v, dict):
            leaf_ns = NameSpace()
            ns.__dict__[k] = leaf_ns
            dump_to_namespace(leaf_ns, v)
        else:
            ns.__dict__[k] = v

config = NameSpace()
dump_to_namespace(config, config_dict)

Answer 7

To extend Thomas's idea, the following preserves type of any supported iterable object that supports list-to-type conversion.

import types
import sys

class RecursiveNamespace(types.SimpleNamespace):
    def __init__(self, accepted_iter_types = [], **kwargs):
        self.supported_types = [list, tuple, set] + accepted_iter_types

        for key, val in kwargs.items():
            if type(val) == dict:
                setattr(self, key, RecursiveNamespace(**val))
            elif hasattr(val, '__iter__'): # object is iterable
                setattr(self, key, self.__make_iterable(val))
            else: 
                setattr(self, key, val)

    def __make_iterable(self, val):
        if(type(val) not in self.supported_types): # not a supoorted iterable type
            return val
        lst = [self.__recurse_in(v) for v in val]
        try:
            ret = type(val)(lst)  # the type is assumed to support list-to-type conversion
        except Exception as e:
            print(f"Failed to make iterable object of type {type(val)}", e, out=sys.stderr)
        return  ret 

    def __recurse_in(self, val):
        if type(val) == dict:
            return RecursiveNamespace(**val)
        elif(hasattr(val, '__iter__')): # if it's iterable
            return self.__make_iterable(val)        
        else:
            return val

    def __getitem__(self, key):
        return self.__dict__[key]

if __name__ == '__main__':
    data = {'a': 1, 
        'b': (2,3), 
        'c': [4,5],
        'd': set([6,'7',8]),
        'e': {
            'e_1': 9, 
            'e_2': {
                    'e_2_1': 10, 
                    'e_2_2': (11,)
                    },
            'e_3': [12,13]}
    }

    rname = RecursiveNamespace(**data)
    print(rname)
    print('%20s :'%(type(rname.a)), rname.a)
    print('%20s :'%(type(rname.b)), rname.b)
    print('%20s :'%(type(rname.c)), rname.c)
    print('%20s :'%(type(rname.d)), rname.d)
    print('%20s :'%(type(rname.e.e_2.e_2_2)), rname.e.e_2.e_2_2)

output:

   <class 'int'> : 1
 <class 'tuple'> : (2, 3)
  <class 'list'> : [4, 5]
   <class 'set'> : {8, '7', 6}
 <class 'tuple'> : (11,)

Answer 8

There's Always this option, I don't know that it is the best method out there, but it sure does work. Assuming type(x) = dict

for key, val in x.items():  # unpack the keys from the dictionary to individual variables
    exec (key + '=val')