Python: globals().items() iterations try to change a dict

Question

In cso.data.__init__.py I'm trying to do following:

from  cso.data._features  import * # import sumbodule 

for k, v in globals().items():
    if isinstance(v, entity):# entity is some type
        print(k,v)

To my surprise, I get:

Traceback (most recent call last):
  File "X:\Programming\workspaceEclipse\PyCommonSence\src\cso\data\__init__.py", line 20, in <module>
    from  cso.data._features  import *
  File "X:\Programming\workspaceEclipse\PyCommonSence\src\cso\data\__init__.py", line 49, in <module>
    for k, v in globals().items():
RuntimeError: dictionary changed size during iteration

For some reason it tries to modify the iterated globals somehow. How can it be ? Is there another better way to list all variables in the form name: value from all modules in the package?

The list comprehension equivalent works without error, but prints twice:

print([ (k, v) for k, v in globals().items() if isinstance(v, entity)])

What is happening here?

i dont want to create new list since i expect there will be lots of items — snamef, Aug 17 '19 at 01:40

ShadowRanger · Accepted Answer · 2019-08-17T01:58:58.827

When you run code at global scope, the names you assign to are globals too. k and v are first assigned after you begin iterating globals().items(), which adds entries for 'k' and 'v' in the globals() dictionary.

You could avoid this by explicitly copying the dict before iterating it, e.g.:

for k, v in globals().copy().items():

or by doing the work in a function (which will store the iteration variables in local scope):

def print_global_entities():
    for k, v in globals().items():
        if isinstance(v, entity):# entity is some type
            print(k,v)

if __name__ == '__main__':
    print_global_entities()

Doing this actually makes your code faster to boot; outside a function, every store to, and load from, k and v involves a dict lookup, while inside a function, Python can (and the CPython reference interpreter does) use simple C array lookups with the indices computed at compile time.

The third approach is the cheaty approach: Make sure k and v exist beforehand, so the globals() dict doesn't add or remove items. All you need for that is to add:

k = v = None

just before the for loop.

yes thats the answer.k = v = None is cool :D Also any ideas why print([ (k, v) for k, v in globals().items() if isinstance(v, entity)]) called TWICE? — snamef, Aug 17 '19 at 01:49
@snamef: It shouldn't, unless you actually run it twice. But you only showed a single line of code, and I can't tell what else you might have done to cause it (or cause something like it, e.g. accidentally nesting a copy of `globals()` into a global that would look almost like a doubled output without newlines to break it up separately). — ShadowRanger, Aug 17 '19 at 01:58

juanpa.arrivillaga · Answer 2 · 2019-08-17T01:40:27.317

Because a for-loop statement creates a new variable(s), in your specific case:

for k, v in globals().items():
    ...

You've created k and v in the global namespace, which is literally that dict

List comprehensions, on the other hand, create their own local function scope. Just do:

for k, v in globals().copy().items():
    ...

If you don't want to copy, then do:

k,v = None, None
for k,v in globals().items():
    ...

Or better yet, wrap this all in a function so you don't accidentally touch the global namespace

Python: globals().items() iterations try to change a dict

2 Answers2

Linked