182

In python 2.7, we got the dictionary view methods available.

Now, I know the pro and cons of the following:

  • dict.items() (and values, keys): returns a list, so you can actually store the result, and
  • dict.iteritems() (and the like): returns a generator, so you can iterate over each value generated one by one.

What are dict.viewitems() (and the like) for? What are their benefits? How does it work? What is a view after all?

I read that the view is always reflecting the changes from the dictionary. But how does it behave from the perf and memory point of view? What are the pro and cons?

AndyB
  • 128
  • 3
Bite code
  • 578,959
  • 113
  • 301
  • 329

5 Answers5

187

Dictionary views are essentially what their name says: views are simply like a window on the keys and values (or items) of a dictionary. Here is an excerpt from the official documentation for Python 3:

>>> dishes = {'eggs': 2, 'sausage': 1, 'bacon': 1, 'spam': 500}
>>> keys = dishes.keys()
>>> values = dishes.values()

>>> # view objects are dynamic and reflect dict changes
>>> del dishes['eggs']
>>> keys  # No eggs anymore!
dict_keys(['sausage', 'bacon', 'spam'])

>>> values  # No eggs value (2) anymore!
dict_values([1, 1, 500])

(The Python 2 equivalent uses dishes.viewkeys() and dishes.viewvalues().)

This example shows the dynamic character of views: the keys view is not a copy of the keys at a given point in time, but rather a simple window that shows you the keys; if they are changed, then what you see through the window does change as well. This feature can be useful in some circumstances (for instance, one can work with a view on the keys in multiple parts of a program instead of recalculating the current list of keys each time they are needed)—note that if the dictionary keys are modified while iterating over the view, how the iterator should behave is not well defined, which can lead to errors.

One advantage is that looking at, say, the keys uses only a small and fixed amount of memory and requires a small and fixed amount of processor time, as there is no creation of a list of keys (Python 2, on the other hand, often unnecessarily creates a new list, as quoted by Rajendran T, which takes memory and time in an amount proportional to the length of the list). To continue the window analogy, if you want to see a landscape behind a wall, you simply make an opening in it (you build a window); copying the keys into a list would correspond to instead painting a copy of the landscape on your wall—the copy takes time, space, and does not update itself.

To summarize, views are simply… views (windows) on your dictionary, which show the contents of the dictionary even after it changes. They offer features that differ from those of lists: a list of keys contain a copy of the dictionary keys at a given point in time, while a view is dynamic and is much faster to obtain, as it does not have to copy any data (keys or values) in order to be created.

Eric O. Lebigot
  • 91,433
  • 48
  • 218
  • 260
  • 7
    +1. Ok, how does that differ from having directly access to the internal list of keys ? Is that faster, slower ? More memory efficient ? Restricted ? If you can read and edit it, it feels exactly the same as having a reference to this list. – Bite code Jan 22 '12 at 13:01
  • 5
    Thanks. The thing is that views *are* your access to "the internal list of keys" (note that this "list of keys" is not a Python list, though, but is precisely a view). Views are more memory efficient than the lists of keys (or values or items) of Python 2, since they don't copy anything; they're indeed like "a reference to the list of keys" (note too that "a reference to a list" is actually simply called a list, in Python, as lists are mutable objects). Also note that you can't directly edit views: instead, your still edit the dictionary, and the views reflect your changes immediately. – Eric O. Lebigot Jan 22 '12 at 13:17
  • 4
    Ok, I'm not clear on the implementation yet, but it's the best answer so far. – Bite code Jan 22 '12 at 14:54
  • 2
    Thanks. Indeed, this answer is mostly about the *semantics* of views. I don't have information about their implementation in CPython, but I would guess that a view is basically a pointer to the right structure(s) (keys and/or values), and that the structures are part of the dictionary object itself. – Eric O. Lebigot Jan 22 '12 at 17:05
  • 5
    I think it's worth pointing out that example code in this post is from python3 and is not what I get in python2.7. – snth Nov 15 '12 at 10:22
  • 1
    Is there anything that we can do with views but not with generators? – gkcn Oct 09 '13 at 12:26
  • 1
    Yes, there are: you can take the length `len()` of a view but not of a generator. In fact, views have a finite number of elements, whereas generators do not have this limitation. That said, they are both memory-efficient structures. They are so in a different way, though: the view does not create additional memory (unlike `dict.keys()` in Python 2), while a generator sometimes does not take any memory at all (up to some negligible amount). Here is a partial reference: http://docs.python.org/dev/library/stdtypes.html#dict-views. – Eric O. Lebigot Oct 10 '13 at 17:21
  • 1
    @snth: in python2.7 it would read `keys = dishes.viewkeys()` and `values = dishes.viewvalues()` – Antony Hatchkins Oct 24 '14 at 09:24
  • 1
    I still lack a real-world example where they could be useful (and in >20 years of Python programming I never encountered one, so please help enlightening me :) What I can do with them I also can do with the dictionary itself (e. g. `len()` of it, iterate over it, ...). So in what actual usecase would I be happy to have these methods? – Alfe Sep 18 '17 at 10:08
  • That's a good question. There is a clear use case: iterating over the values of a dictionary; in fact, it is more efficiently done when the values are not copied. This is what every single `dict.values()` provides in Python 3, where it returns a view. Now, as for the dynamic aspect of the view, I must say that I indeed don't recall the need for it. I'd be curious to see a use case. Maybe that's a cheap side effect that was deemed potentially useful? – Eric O. Lebigot Sep 19 '17 at 05:21
  • This is a great answer. The only thing that's missing is some clarification on how the "views automatically update" feature can be used usefully even though, according to Python docs, "Iterating views while adding or deleting entries in the dictionary may raise a RuntimeError or fail to iterate over all entries." (https://docs.python.org/release/3.3.0/library/stdtypes.html#dict-views). It seems like that would really limit the usefulness of auto-updating but maybe there are some ways to avoid problems. But regardless, the point about CPU/memory is clear. – Stephen Apr 24 '18 at 20:24
  • Thanks. Iterating over a view and modifying its dictionary keys during the iteration cannot be a well defined operation (try to define what keys the iteration should return!), so using the "automatic update of views" in this circumstances is not something anybody should want to do. Now, to elaborate on the usage I was delineating, you can imagine a dictionary with keys as student IDs: you could store the set of IDs as a view on this dictionary. Thus, the _current_ list of IDs, after any dictionary update, can be printed at any time by iterating over this view. Does this make more sense? – Eric O. Lebigot Apr 24 '18 at 20:39
23

Just from reading the docs I get this impression:

  1. Views are "pseudo-set-like", in that they don't support indexing, so what you can do with them is test for membership and iterate over them (because keys are hashable and unique, the keys and items views are more "set-like" in that they don't contain duplicates).
  2. You can store them and use them multiple times, like the list versions.
  3. Because they reflect the underlying dictionary, any change in the dictionary will change the view, and will almost certainly change the order of iteration. So unlike the list versions, they're not "stable".
  4. Because they reflect the underlying dictionary, they're almost certainly small proxy objects; copying the keys/values/items would require that they watch the original dictionary somehow and copy it multiple times when changes happen, which would be an absurd implementation. So I would expect very little memory overhead, but access to be a little slower than directly to the dictionary.

So I guess the key usecase is if you're keeping a dictionary around and repeatedly iterating over its keys/items/values with modifications in between. You could just use a view instead, turning for k, v in mydict.iteritems(): into for k, v in myview:. But if you're just iterating over the dictionary once, I think the iter- versions are still preferable.

Ben
  • 68,572
  • 20
  • 126
  • 174
  • 2
    +1 for analyzing the pro and cons from the few informations we got. – Bite code Jan 22 '12 at 10:03
  • If I create an iterator over a view, it still gets invalidated whenever the dictionary changes. That's the same trouble as with an iterator over the dictionary itself (e. g. `iteritems()`). So what's the point of these views? When am I happy to have them? – Alfe Sep 18 '17 at 10:11
  • @Alfe You're right, that is a problem with dictionary iteration and views don't help with it at all. Say you need to pass a dictionary's values to a function. You could use `.values()`, but that involves making a whole copy as a list, which could be expensive. There's `.itervalues()` but you can't consume those more than once, so it won't work with every function. Views don't require an expensive copy, but they're still more useful as a stand alone value than an iterator. But they're still not intended to help with iterating and modifying at the same time (there you really do want a copy). – Ben Sep 18 '17 at 23:01
22

As you mentioned dict.items() returns a copy of the dictionary’s list of (key, value) pairs which is wasteful and dict.iteritems() returns an iterator over the dictionary’s (key, value) pairs.

Now take the following example to see the difference between an interator of dict and a view of dict

>>> d = {"x":5, "y":3}
>>> iter = d.iteritems()
>>> del d["x"]
>>> for i in iter: print i
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration

Whereas a view simply shows you what's in the dict. It doesn't care if it changed:

>>> d = {"x":5, "y":3}
>>> v = d.viewitems()
>>> v
dict_items([('y', 3), ('x', 5)])
>>> del d["x"]
>>> v
dict_items([('y', 3)])

A view is simply a what the dictionary looks like now. After deleting an entry .items() would have been out-of-date and .iteritems() would have thrown an error.

Martin Konecny
  • 57,827
  • 19
  • 139
  • 159
15

The view methods return a list(not a copy of the list, compared to .keys(), .items() and .values()), so it is more lightweight, but reflects the current contents of dictionary.

From Python 3.0 - dict methods return views - why?

The main reason is that for many use cases returning a completely detached list is unnecessary and wasteful. It would require copying the entire content (which may or many not be a lot).

If you simply want to iterate over the keys then creating a new list is not necessary. And if you indeed need it as a separate list (as a copy) then you can easily create that list from the view.

Community
  • 1
  • 1
Rajendran T
  • 1,513
  • 10
  • 15
8

Views let you access the underlaying data structure, without copying it. Besides being dynamic as opposed to creating a list, one of their most useful usage is in test. Say you want to check if a value is in the dict or not (either it be key or value).

Option one is to create a list of the keys using dict.keys(), this works but obviously consumes more memory. If the dict is very large? That would be wasteful.

With views you can iterate the actual data-structure, without intermediate list.

Let's use examples. I've a dict with 1000 keys of random strings and digits and k is the key I want to look for

large_d = { .. 'NBBDC': '0RMLH', 'E01AS': 'UAZIQ', 'G0SSL': '6117Y', 'LYBZ7': 'VC8JQ' .. }

>>> len(large_d)
1000

# this is one option; It creates the keys() list every time, it's here just for the example
timeit.timeit('k in large_d.keys()', setup='from __main__ import large_d, k', number=1000000)
13.748743600954867


# now let's create the list first; only then check for containment
>>> list_keys = large_d.keys()
>>> timeit.timeit('k in list_keys', setup='from __main__ import large_d, k, list_keys', number=1000000)
8.874809793833492


# this saves us ~5 seconds. Great!
# let's try the views now
>>> timeit.timeit('k in large_d.viewkeys()', setup='from __main__ import large_d, k', number=1000000)
0.08828549011070663

# How about saving another 8.5 seconds?

As you can see, iterating view object gives a huge boost to performance, reducing memory overhead at the same time. You should use them when you need to perform Set like operations.

Note: I'm running on Python 2.7

Chen A.
  • 10,140
  • 3
  • 42
  • 61
  • In python >=3, I believe `.keys()` returns a view by default. Might want to double check tho – Yolo Voe Jul 25 '18 at 01:57
  • 1
    You're right. Python 3+ do heavy usage of view objects instead of lists, it is much more memory efficient – Chen A. Jul 25 '18 at 07:57
  • 1
    These timing results are very telling, but checking whether `k` is one of the keys of dictionary `large_d` is meant to be done with `k in large_d`, in Python, which is probably essentially as fast as using a view (in other words, `k in large_d.keys()` is not Pythonic and should be avoided—as is `k in large_d.viewkeys()`). – Eric O. Lebigot Aug 18 '18 at 20:51
  • Thank you for providing a solid, useful example. `k in large_d` is actually significantly faster than `k in large_d.viewkeys()`, so that should probably be avoided, but this makes sense for `k in large_d.viewvalues()`. – naught101 Nov 27 '18 at 22:19