9

I've encountered an issue re scopes in a lambda function. I can successfully output foo to stdout but I get an error when using max() including a lambda - see simplified code below...

All in all, I am trying find the largest value for a nested key budget within an unknown number of first order keys.

(Pdb) foo = self.some_method()    # some_method() returns a dict, printed in the next step

(Pdb) pp foo

{'1': {'count': 1,
       'extra_data': {'activity-count': 1,
                             'budget': 0,
                             [...MORE KEY-VALUE PAIRS HERE...]
                             'version': 1},
       [...LOTS MORE KEY-VALUE PAIRS HERE...]
       'elements_total': defaultdict(<type 'int'>, {'result': 1, 'another_key': 2}),
       'extra_year_data': defaultdict(<function <lambda> at 0x10e05bd70>, {})}, 

 '2': {'count': 1,
       'extra_data': {'activity-count': 1,
                             'budget': 3,
                             [...MORE KEY-VALUE PAIRS HERE...]
                             'version': 1},
       [...LOTS MORE KEY-VALUE PAIRS HERE...]
       'elements_total': defaultdict(<type 'int'>, {'result': 1, 'another_key': 2}),
       'extra_year_data': defaultdict(<function <lambda> at 0x10e05bd70>, {})}}

(Pdb) max(foo, key=lambda x: foo[x]['extra_data']['budget'])
*** NameError: global name 'foo' is not defined

All in all, I am trying to use max(foo, key=lambda x: foo[x]['extra_data']['budget']) to find the largest value for a nested key budget within an unknown number of first order keys.

The expected result in this case could be 2 as the value for foo['2']['extra_data']['budget'] = 3 vs. foo['1']['extra_data']['budget'] = 0.

Could the error be related to the fact that some of the (unrelated) keys have defaultdicts within them?

smci
  • 32,567
  • 20
  • 113
  • 146
user2761030
  • 1,385
  • 2
  • 20
  • 33

3 Answers3

11

You set a new local with pdb, but that is not visible to expressions using nested scopes in this debugger session. Any expression in a nested scope such as the lambda used for the key argument, using a name that is local to the current frame, would need to be a closure and will have this problem.

That's a limitation of how the debugger and Python compilation work; closures can only be created if the function that need to produce them was compiled in the same session. Since the function you are debugging was compiled without foo being a closure, it cannot be used by the lambda expression as such.

You can bind the local to the lambda (making it a local rather than a closure):

max(foo, key=lambda x, foo=foo: foo[x]['extra_data']['budget'])

See What exactly is contained within a obj.__closure__? for details on how the Python compiler creates closures.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Thanks, I think that I understand this now. Does this mean that when I add this to my script, I don't need to bind the local to the lambda - i.e. I can just use `max(foo, key=lambda x: foo[x]['extra_data']['budget'])` ? – user2761030 Feb 02 '16 at 11:35
  • @user2761030: sorry, trying again. Yes, you can add this to your script, and because the Python compiler then sees the name `foo` in the `lambda` being used it'll create the closure in the parent scope. It'll then work. You just can't do this in `pdb`. – Martijn Pieters Feb 02 '16 at 11:38
8

There is a bug report for Python 3 (however this issue affects Python 2.7 as well as you found out) which suggests a workaround as an alternative to Martijn's solution: interact at the pdb prompt drops you into an interactive session which is populated with globals() and locals() and your lambda should work as expected.

kynan
  • 13,235
  • 6
  • 79
  • 81
6

This is gonna mess with your global scope but it's a quick (dirty) workaround I use in this case when using python 2.7:

globals().update(locals())
olivecoder
  • 2,858
  • 23
  • 22