139

Let's say I've got a simple class in python

class Wharrgarbl(object):
    def __init__(self, a, b, c, sum, version='old'):
        self.a = a
        self.b = b
        self.c = c
        self.sum = 6
        self.version = version

    def __int__(self):
        return self.sum + 9000

    def __what_goes_here__(self):
        return {'a': self.a, 'b': self.b, 'c': self.c}

I can cast it to an integer very easily

>>> w = Wharrgarbl('one', 'two', 'three', 6)
>>> int(w)
9006

Which is great! But, now I want to cast it to a dict in a similar fashion

>>> w = Wharrgarbl('one', 'two', 'three', 6)
>>> dict(w)
{'a': 'one', 'c': 'three', 'b': 'two'}

What do I need to define for this to work? I tried substituting both __dict__ and dict for __what_goes_here__, but dict(w) resulted in a TypeError: Wharrgarbl object is not iterable in both cases. I don't think simply making the class iterable will solve the problem. I also attempted many googles with as many different wordings of "python cast object to dict" as I could think of but couldn't find anything relevant :{

Also! Notice how calling w.__dict__ won't do what I want because it's going to contain w.version and w.sum. I want to customize the cast to dict in the same way that I can customize the cast to int by using def int(self).

I know that I could just do something like this

>>> w.__what_goes_here__()
{'a': 'one', 'c': 'three', 'b': 'two'}

But I am assuming there is a pythonic way to make dict(w) work since it is the same type of thing as int(w) or str(w). If there isn't a more pythonic way, that's fine too, just figured I'd ask. Oh! I guess since it matters, this is for python 2.7, but super bonus points for a 2.4 old and busted solution as well.

There is another question Overloading __dict__() on python class that is similar to this one but may be different enough to warrant this not being a duplicate. I believe that OP is asking how to cast all the data in his class objects as dictionaries. I'm looking for a more customized approach in that I don't want everything in __dict__ included in the dictionary returned by dict(). Something like public vs private variables may suffice to explain what I'm looking for. The objects will be storing some values used in calculations and such that I don't need/want to show up in the resulting dictionaries.

UPDATE: I've chosen to go with the asdict route suggested but it was a tough choice selecting what I wanted to be the answer to the question. Both @RickTeachey and @jpmc26 provided the answer I'm going to roll with but the former had more info and options and landed on the same result as well and was upvoted more so I went with it. Upvotes all around though and thanks for the help. I've lurked long and hard on stackoverflow and I'm trying to get my toes in the water more.

Community
  • 1
  • 1
penchant
  • 2,263
  • 3
  • 14
  • 20
  • 1
    have you seen this already? https://docs.python.org/3/reference/datamodel.html?emulating-container-types#emulating-container-types – Garrett R Feb 09 '16 at 01:04
  • 1
    @GarrettR It's helpful, but it doesn't really reveal a lot of insight into this particular issue. – Zizouz212 Feb 09 '16 at 01:06
  • Possible duplicate of [Overloading \_\_dict\_\_() on python class](http://stackoverflow.com/questions/23252370/overloading-dict-on-python-class) – Rick Feb 09 '16 at 14:08
  • @RickTeachey it certainly is similar to the Overloading... question, although I am looking for a more customizable dict representation. For example the object may be storing things I don't care to see in the dict. – penchant Feb 09 '16 at 21:12
  • Your question is misleadingly phrased, you don't want to convert or cast your object to dict representation (such that you could completely recover the object if you converted back). You only want to create a dict representation view for (say) public instance variables. – smci Jan 15 '20 at 15:39

10 Answers10

218

There are at least five six ways. The preferred way depends on what your use case is.

Option 1:

Simply add an asdict() method.

Based on the problem description I would very much consider the asdict way of doing things suggested by other answers. This is because it does not appear that your object is really much of a collection:

class Wharrgarbl(object):

    ...

    def asdict(self):
        return {'a': self.a, 'b': self.b, 'c': self.c}

Using the other options below could be confusing for others unless it is very obvious exactly which object members would and would not be iterated or specified as key-value pairs.

Option 1a:

Inherit your class from 'typing.NamedTuple' (or the mostly equivalent 'collections.namedtuple'), and use the _asdict method provided for you.

from typing import NamedTuple

class Wharrgarbl(NamedTuple):
    a: str
    b: str
    c: str
    sum: int = 6
    version: str = 'old'

Using a named tuple is a very convenient way to add lots of functionality to your class with a minimum of effort, including an _asdict method. However, a limitation is that, as shown above, the NT will include all the members in its _asdict.

If there are members you don't want to include in your dictionary, you'll need to specify which members you want the named tuple _asdict result to include. To do this, you could either inherit from a base namedtuple class using the older collections.namedtuple API:

from collections import namedtuple as nt

class Wharrgarbl(nt("Basegarble", "a b c")):
    # note that the typing info below isn't needed for the old API
    a: str
    b: str
    c: str
    sum: int = 6
    version: str = 'old'

...or you could create a base class using the newer API, and inherit from that, using only the dictionary members in the base class:

from typing import NamedTuple

class Basegarbl(NamedTuple):
    a: str
    b: str
    c: str

class Wharrgarbl(Basegarbl):
    sum: int = 6
    version: str = 'old'

Another limitation is that NT is read-only. This may or may not be desirable.

Option 2:

Implement __iter__.

Like this, for example:

def __iter__(self):
    yield 'a', self.a
    yield 'b', self.b
    yield 'c', self.c

Now you can just do:

dict(my_object)

This works because the dict() constructor accepts an iterable of (key, value) pairs to construct a dictionary. Before doing this, ask yourself the question whether iterating the object as a series of key,value pairs in this manner- while convenient for creating a dict- might actually be surprising behavior in other contexts. E.g., ask yourself the question "what should the behavior of list(my_object) be...?"

Additionally, note that accessing values directly using the get item obj["a"] syntax will not work, and keyword argument unpacking won't work. For those, you'd need to implement the mapping protocol.

Option 3:

Implement the mapping protocol. This allows access-by-key behavior, casting to a dict without using __iter__, and also provides two types of unpacking behavior:

  1. mapping unpacking behavior: {**my_obj}
  2. keyword unpacking behavior, but only if all the keys are strings: dict(**my_obj)

The mapping protocol requires that you provide (at minimum) two methods together: keys() and __getitem__.

class MyKwargUnpackable:
    def keys(self):
        return list("abc")
    def __getitem__(self, key):
        return dict(zip("abc", "one two three".split()))[key]

Now you can do things like:

>>> m=MyKwargUnpackable()
>>> m["a"]
'one'
>>> dict(m)  # cast to dict directly
{'a': 'one', 'b': 'two', 'c': 'three'}
>>> dict(**m)  # unpack as kwargs
{'a': 'one', 'b': 'two', 'c': 'three'}

As mentioned above, if you are using a new enough version of python you can also unpack your mapping-protocol object into a dictionary comprehension like so (and in this case it is not required that your keys be strings):

>>> {**m}
{'a': 'one', 'b': 'two', 'c': 'three'}

Note that the mapping protocol takes precedence over the __iter__ method when casting an object to a dict directly (without using kwarg unpacking, i.e. dict(m)). So it is possible- and might be sometimes convenient- to cause the object to have different behavior when used as an iterable (e.g., list(m)) vs. when cast to a dict (dict(m)).

But note also that with regular dictionaries, if you cast to a list, it will give the KEYS back, and not the VALUES as you require. If you implement another nonstandard behavior for __iter__ (returning values instead of keys), it could be surprising for other people using your code unless it is very obvious why this would happen.

EMPHASIZED: Just because you CAN use the mapping protocol, does NOT mean that you SHOULD do so. Does it actually make sense for your object to be passed around as a set of key-value pairs, or as keyword arguments and values? Does accessing it by key- just like a dictionary- really make sense? Would you also expect your object to have other standard mapping methods such as items, values, get? Do you want to support the in keyword and equality checks (==)?

If the answer to these questions is yes, it's probably a good idea to not stop here, and consider the next option instead.

Option 4:

Look into using the 'collections.abc' module.

Inheriting your class from 'collections.abc.Mapping or 'collections.abc.MutableMapping signals to other users that, for all intents and purposes, your class is a mapping * and can be expected to behave that way. It also provides the methods items, values, get and supports the in keyword and equality checks (==) "for free".

You can still cast your object to a dict just as you require, but there would probably be little reason to do so. Because of duck typing, bothering to cast your mapping object to a dict would just be an additional unnecessary step the majority of the time.

This answer from me about how to use ABCs might also be helpful.

As noted in the comments below: it's worth mentioning that doing this the abc way essentially turns your object class into a dict-like class (assuming you use MutableMapping and not the read-only Mapping base class). Everything you would be able to do with dict, you could do with your own class object. This may be, or may not be, desirable.

Also consider looking at the numerical abcs in the numbers module:

https://docs.python.org/3/library/numbers.html

Since you're also casting your object to an int, it might make more sense to essentially turn your class into a full fledged int so that casting isn't necessary.

Option 5:

Look into using the dataclasses module (Python 3.7+ only), which includes a convenient asdict() utility method.

from dataclasses import dataclass, asdict, field, InitVar

@dataclass
class Wharrgarbl(object):
    a: int
    b: int
    c: int
    sum: InitVar[int]  # note: InitVar will exclude this from the dict
    version: InitVar[str] = "old"

    def __post_init__(self, sum, version):
        self.sum = 6  # this looks like an OP mistake?
        self.version = str(version)

Now you can do this:

    >>> asdict(Wharrgarbl(1,2,3,4,"X"))
    {'a': 1, 'b': 2, 'c': 3}

Option 6:

Use typing.TypedDict, which has been added in python 3.8.

NOTE: option 6 is likely NOT what the OP, or other readers based on the title of this question, are looking for. See additional comments below.

class Wharrgarbl(TypedDict):
    a: str
    b: str
    c: str

Using this option, the resulting object is a dict (emphasis: the object class will be dict, not Wharrgarbl). There is no reason at all to "cast" it to a dict (unless you are making a copy) because it already is.

And since the object is a dict, the initialization signature is identical to that of dict and as such it only accepts keyword arguments or another dictionary.

    >>> w = Wharrgarbl(a=1,b=2,b=3)
    >>> w
    {'a': 1, 'b': 2, 'c': 3}
    >>> type(w)
    <class 'dict'>

Emphasized: the above "class" Wharrgarbl isn't actually a new class at all. It is simply syntactic sugar for creating typed dict objects with specific keys ONLY and value fields of different types for the type checker. At run time, it is still nothing more than a dict.

This option can be pretty convenient for signaling to readers of your code (and also to a type checker such as mypy) that these particular dict objects expected to have specific keys with specific value types.

But this means you cannot, for example, add other methods, although you can try:

class MyDict(TypedDict):
    def my_fancy_method(self):
        return "world changing result"

...but it won't work:

>>> MyDict().my_fancy_method()
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
AttributeError: 'dict' object has no attribute 'my_fancy_method'

* "Mapping" has become the standard "name" of the dict-like duck type

Rick
  • 43,029
  • 15
  • 76
  • 119
  • 1
    This *is* a way to get the effect the OP wants, but you should maybe clarify that it has other effects, e.g. `for k, v in my_object` will also be enabled. – zwol Feb 09 '16 at 03:16
  • @RickTeachey I see what you are driving at with the asdict point. I certainly don't need the objects to be iterable. – penchant Feb 09 '16 at 21:10
  • Why we can's simply use vars function as mentioned in other linkhttps://stackoverflow.com/questions/61517/python-dictionary-from-an-objects-fields. Am i missing some thing in this question ? – user Jul 21 '20 at 13:27
  • @user The OP's question wanted only members `a`, `b`, and `c` in the `dict`. The `vars` function will return a dictionary with ALL of the object members including the ones that were not wanted: `sum` and `version`. In some cases `vars` will work but it isn't really idiomatic python in my experience. Usually people are more explicit, saying "I am now casting my object to a dictionary" by doing e.g. `obj.asdict()`. With `vars`, it is usually something more like `obj_kwargs = vars(obj)` followed by `MyObject(**obj_kwargs)`. In other words: `vars` is mostly used to copy objects. – Rick Jul 21 '20 at 13:39
  • how about adding a method that returns `self.__dict__`? – user32882 Nov 03 '20 at 13:11
  • It would also return a dict of the attributes ... Wouldn't it? – user32882 Nov 03 '20 at 14:38
  • @user32882 yes it does-- of ALL the attributes. which might work in some instances. but `__dict__` is not part of the python dictionary data structure API. What you are probably looking for is either the mapping protocol (see option 3 above) or addition of an asdict method (option 1 above). – Rick Nov 03 '20 at 15:00
  • 1
    Comment on Option 3: while `dict` accepts an arbitrary iterable of key/value pairs, that doesn't necessarily mean that `__iter__` *must* yield such pairs. When it makes sense to do so, an object can be iterated over in a way that `dict` accepts, but you can define `__iter__` differently. (`list`, for example, produces a `listiterator` value that is *not* compatible with the format `dict` expects.) – chepner Jan 03 '22 at 17:02
  • @chepner thank you; yes I supplied comments to this effect under #2, I believe: "`...ask yourself the question whether iterating the object as a series of key,value pairs in this manner- while convenient for creating a dict- might actually be surprising behavior in other contexts. E.g., ask yourself the question "what should the behavior of \`list(my_object)\` be...?"`". But it might be worth a short reiteration under #3, I'll think about doing an edit. – Rick Jan 03 '22 at 21:37
  • 1
    Option 1a does not work with python 3.6. I get "AttributeError: Cannot overwrite NamedTuple attribute _asdict". Was this something that is now no longer possible, or something that was introduced later? – ljden Mar 08 '22 at 05:38
  • @ljden Thanks for pointing this out! The named tuple APIs (both the old and new API) do allow for overriding in a child class so back when I wrote this I think I just assumed it to be ok for overriding in a `NamedTuple` (without checking). I'll correct the error. – Rick Mar 08 '22 at 15:41
22

There is no magic method that will do what you want. The answer is simply name it appropriately. asdict is a reasonable choice for a plain conversion to dict, inspired primarily by namedtuple. However, your method will obviously contain special logic that might not be immediately obvious from that name; you are returning only a subset of the class' state. If you can come up with with a slightly more verbose name that communicates the concepts clearly, all the better.

Other answers suggest using __iter__, but unless your object is truly iterable (represents a series of elements), this really makes little sense and constitutes an awkward abuse of the method. The fact that you want to filter out some of the class' state makes this approach even more dubious.

jpmc26
  • 28,463
  • 14
  • 94
  • 146
  • yes, returning a subset of the class's whole state, the values of relevance, is what I'm trying to achieve. I've got some "hidden" values stored that make things quicker for calculations/comparisons etc, but they are nothing that a human would want to see or play with and they also don't really represent the object itself. – penchant Feb 09 '16 at 21:25
  • @tr3buchet In your example code, those values are *passed in to the initializer*. Is that the way it is in your real code? It seems odd that cached calculations would be passed in as arguments. – jpmc26 Feb 13 '16 at 07:34
  • not at all, I was just doing it to provide an example – penchant Feb 19 '16 at 21:55
  • Just want to note that the [docs](https://docs.python.org/2/reference/datamodel.html#object.__iter__) seem to condone using `__iter__` as other answers have suggested: "For mappings, it should iterate over the keys of the container, and should also be made available as the method iterkeys()." – dbenton Feb 24 '18 at 23:19
  • @dbenton That is not what the OP is doing or what the other answers are recommending. The OP's object is not a "container," which in this context is basically being used as a synonym of collection. The OP's object is also not what is usually meant by mapping. A mapping would be something that has an arbitrary number of keys and values and should extend the [`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) base class. An object that has specific named attributes does not fall into this category. – jpmc26 Feb 24 '18 at 23:24
13

something like this would probably work

class MyClass:
    def __init__(self,x,y,z):
       self.x = x
       self.y = y
       self.z = z
    def __iter__(self): #overridding this to return tuples of (key,value)
       return iter([('x',self.x),('y',self.y),('z',self.z)])

dict(MyClass(5,6,7)) # because dict knows how to deal with tuples of (key,value)
Joran Beasley
  • 110,522
  • 12
  • 160
  • 179
  • oh interesting! an iterable of the tuples, I didn't consider this. I'll do a quick test – penchant Feb 09 '16 at 01:09
  • Also, returning an iter of a list of tuples snapshots all the values of x, y, and z into the list on which the iter is based. In the case if the 3 yield statements, it is possible (a tiny window, but there nonetheless) that running this in a threaded environment could result in another thread modifying the y attribute or the z attribute such that the reported x, y, and z are not consistent. – PaulMcG Feb 09 '16 at 02:11
  • @PaulMcGuire: does CPython guarante that they'll all be consistent even with Joran's code? I would have thought that another thread could be scheduled, for example, after the first tuple with `x` is created, but before the second tuple with `y`. Am I wrong, and actually the list comprehension is atomic with respect to thread scheduling? – Steve Jessop Feb 09 '16 at 12:57
  • Now that you ask, I realize that even the list comprehension is not guaranteed atomic, as Python's thread switcher does so at the bytecode level, not the source level. But your exposure is only for the duration of creating the list - once the list has been created, then any updates to the the properties won't matter (unless the property values are mutable objects). Best to use the `dis` module to view the actual bytecodes. – PaulMcG Feb 09 '16 at 19:57
12

I think this will work for you.

class A(object):
    def __init__(self, a, b, c, sum, version='old'):
        self.a = a
        self.b = b
        self.c = c
        self.sum = 6
        self.version = version

    def __int__(self):
        return self.sum + 9000

    def __iter__(self):
        return self.__dict__.iteritems()

a = A(1,2,3,4,5)
print dict(a)

Output

{'a': 1, 'c': 3, 'b': 2, 'sum': 6, 'version': 5}

Garrett R
  • 2,687
  • 11
  • 15
  • what happened to sum and version? how is this any different than just accessing `__dict__` like the other answer suggested ? – Joran Beasley Feb 09 '16 at 01:18
  • it's not dict. the call to iteritems returns a iterator – Garrett R Feb 09 '16 at 01:18
  • 1
    I like this answer ... but it needs a little more work before it meets the OP's requirements ... – Joran Beasley Feb 09 '16 at 01:21
  • 1
    this answer is really no different from @JoranBeasley. The difference is that his is selective whereas your grabs the whole `self.__dict__` which contains items I don't want to have in the dictionary version of the object. – penchant Feb 09 '16 at 21:03
8

Like many others, I would suggest implementing a to_dict() function rather than (or in addition to) allowing casting to a dictionary. I think it makes it more obvious that the class supports that kind of functionality. You could easily implement such a method like this:

def to_dict(self):
    class_vars = vars(MyClass)  # get any "default" attrs defined at the class level
    inst_vars = vars(self)  # get any attrs defined on the instance (self)
    all_vars = dict(class_vars)
    all_vars.update(inst_vars)
    # filter out private attributes
    public_vars = {k: v for k, v in all_vars.items() if not k.startswith('_')}
    return public_vars
tvt173
  • 1,746
  • 19
  • 17
1

It's hard to say without knowing the whole context of the problem, but I would not override __iter__.

I would implement __what_goes_here__ on the class.

as_dict(self:
    d = {...whatever you need...}
    return d
cam
  • 778
  • 6
  • 9
1

I am trying to write a class that is "both" a list or a dict. I want the programmer to be able to both "cast" this object to a list (dropping the keys) or dict (with the keys).

Looking at the way Python currently does the dict() cast: It calls Mapping.update() with the object that is passed. This is the code from the Python repo:

def update(self, other=(), /, **kwds):
    ''' D.update([E, ]**F) -> None.  Update D from mapping/iterable E and F.
        If E present and has a .keys() method, does:     for k in E: D[k] = E[k]
        If E present and lacks .keys() method, does:     for (k, v) in E: D[k] = v
        In either case, this is followed by: for k, v in F.items(): D[k] = v
    '''
    if isinstance(other, Mapping):
        for key in other:
            self[key] = other[key]
    elif hasattr(other, "keys"):
        for key in other.keys():
            self[key] = other[key]
    else:
        for key, value in other:
            self[key] = value
    for key, value in kwds.items():
        self[key] = value

The last subcase of the if statement, where it is iterating over other is the one most people have in mind. However, as you can see, it is also possible to have a keys() property. That, combined with a __getitem__() should make it easy to have a subclass be properly casted to a dictionary:

class Wharrgarbl(object):
    def __init__(self, a, b, c, sum, version='old'):
        self.a = a
        self.b = b
        self.c = c
        self.sum = 6
        self.version = version

    def __int__(self):
        return self.sum + 9000

    def __keys__(self):
        return ["a", "b", "c"]

    def __getitem__(self, key):
        # have obj["a"] -> obj.a
        return self.__getattribute__(key)

Then this will work:

>>> w = Wharrgarbl('one', 'two', 'three', 6)
>>> dict(w)
{'a': 'one', 'c': 'three', 'b': 'two'}
Jérémie
  • 1,353
  • 9
  • 19
  • 1
    You have implemented the mapping protocol. See *Option 3* in my answer above. If you also want to make a `list` dropping the keys, you can add an `__iter__` method that iterates through the values, and the casting to `dict` will still continue to work. Note that usually with dictionaries, if you cast to a list, it will give the KEYS back, and not the values as you require. This could be surprising for other people using your code unless it is obvious why this would happen. – Rick Jun 05 '20 at 15:43
0

Here is very clean and fast solution I created a function that converts any custom class to dict

def convert_to_dict(args: dict):
json = dict()
for key, value in args.items():
    key_vals = str(key).split("_")
    last_index = len(key_vals)
    json[str(key_vals[last_index-1])] = value
return json

what you need is to supply it object.__dict__ then it will do the job for you to clean it and help you store it in databse.

  • 1
    This doesn't do anything for a custom class; it simply turns a `dict` (or technically, anything with an `items` method defined according to the mapping protocol) into another `dict` with modified keys. – chepner Jan 03 '22 at 17:08
  • all you have to do is provide it the solution to the question, then it will repeat that back to you – Justin Jun 13 '22 at 16:52
0

Going off of the asdict solution, here's a useful mixin for if you want to add asdict to several different classes.

Adapted from: https://www.pythontutorial.net/python-oop/python-mixin/

class DictMixin:
    def asdict(self):
         return self._traverse_dict(self._attrs)

    def _traverse_dict(self, attributes: dict) -> dict:
        result = {}
        for key, value in attributes.items():
            result[key] = self._traverse(value)

        return result

    def _traverse(self, value):
        if isinstance(value, DictMixin):
            return value.asdict()
        elif isinstance(value, dict):
            return self._traverse_dict(value)
        elif isinstance(value, list):
            return [self._traverse(v) for v in value]
        else:
            return value


Which you can then use:

class FooBar(DictMixin):
    _attrs = ["foo", "hello"]

    def __init__(self):
        self.foo = "bar"
        self.hello = "world"
>>> a = FooBar()
>>> a.asdict() 
{
    "foo": "bar",
    "hello": "world"
}

Jesse
  • 860
  • 1
  • 11
  • 13
0

You can create a folder like 'Strategy' then you can use pickle to save and load the objects of your class.

import pickle
import os

# Load object as dictionary ---------------------------------------------------
def load_object():
    file_path = 'Strategy\\All_Pickles.hd5'
    if not os.path.isfile(file_path):
        return {}
    with open(file_path, 'rb') as file:
        unpickler = pickle.Unpickler(file)
        return dict(unpickler.load())


# Save object as dictionary ---------------------------------------------------
def save_object(name, value):
    file_path = 'Strategy\\All_Pickles.hd5'
    object_dict = load_object()
    with open(file_path, 'wb') as file:
        object_dict[name] = value
        pickle.dump(object_dict, file)
        return True


class MyClass:
    def __init__(self, name):
        self.name = name

    def show(self):
        print(self.name)


save_object('1', MyClass('Test1'))
save_object('2', MyClass('Test2'))
objects = load_object()
obj1 = objects['1']
obj2 = objects['2']
obj1.show()
obj2.show()

I created two objects of one class and called a method of the class. I hope, it can help you.

Mehdi Hasirchi
  • 236
  • 2
  • 5