How to iterate over dictionary in Python whose (k,v) pair uses consecutive items from a list

Question

I want to add (k,v) pairs to a dictionary weights where the key of this dictionary equals an object from a list layers. The construction of the value is such that it uses object l and object l+1 from this list. I currently do this as follows:

layers = self.layers

for l in range(0, layers.__len__() - 1):
   weights[layers[l]] = np.random.rand(
      layers[l + 1].node_cardinality, 
      layers[l].node_cardinality + 1 
   )

Is there a better, shorter way of doing this without having to use the range() code?

Use for k, v in mydict.items() if you are using python 3,or for k, v in mydict.iteritems() if you are using python 2 — , Mar 04 '19 at 06:40
Possible duplicate of [Iterating over dictionaries using 'for' loops](https://stackoverflow.com/questions/3294889/iterating-over-dictionaries-using-for-loops) — , Mar 04 '19 at 06:41

gilch · Accepted Answer · 2019-03-04T07:26:16.923

Zip with the layers and a slice of layers that begins with the second item. Note that zip() will stop as soon as any of its iterables are exhausted.

for L0, L1 in zip(layers, layers[1:]):
   weights[L0] = np.random.rand(
       L1.node_cardinality, 
       L0.node_cardinality + 1 
   )

Depending on what type of sequence layers is, it may be more efficient to use an itertools.islice instead of a normal slice. A numpy array will probably just use a view if you slice. But a list would have to create a (shallow) copy, so an islice is better if it's very long.

for L0, L1 in zip(layers, islice(layers, 1, None)):
   weights[L0] = np.random.rand(
       L1.node_cardinality, 
       L0.node_cardinality + 1 
   )

As GrazingScientist has pointed out, this could also be done with a dict comprehension.

weights.update(
    {
        L0: np.random.rand(L1.node_cardinality, L0.node_cardinality + 1)
        for L0, L1 in zip(layers, layers[1:])
    }
)

But this approach will have to generate a new dict before updating, which may take more memory. The for loop is probably better if layers is long. But dicts only contain references, so it's probably not much worse than using a normal list slice is.

It would actually be more efficient to use a generator expression in this case. The .update() method can also accept an iterable of (k, v) pairs, but a generator doesn't have to allocate them all at once. It just does a pair at a time.

weights.update(
    (L0, np.random.rand(L1.node_cardinality, L0.node_cardinality + 1))
    for L0, L1 in zip(layers, islice(layers, 1, None))
)

Adrian · Answer 2 · 2019-03-04T06:56:16.410

I would suggest to go with enumerate and a dict comprehension. This is more pythonic and probably faster.

layers = self.layers
buf_dict = {layers[i]: np.random.rand(layers[i + 1].node_cardinality, layers[i].node_cardinality + 1)
for i, l in zip(layers, layers[1:])}
weights.update(buf_dict)

Edit: Forgot code and acknowledged that zip is actually better than enumerate in this case (thanks to gilch), because you won't run into an IndexError.

How to iterate over dictionary in Python whose (k,v) pair uses consecutive items from a list

2 Answers2