123

For the following code:

for sort_key, order in query_data['sort']:
    results.sort(key=lambda k: get_from_dot_path(k, sort_key),
                 reverse=(order == -1))

Pylint reported an error:

Cell variable sort_key defined in loop (cell-var-from-loop)

Could anyone give a hint what is happening here? From pylint source code the description is:

A variable used in a closure is defined in a loop. This will result in all closures using the same value for the closed-over variable.

But I do not have a clue what it means. Could anyone give an example of the problem?

Chris Martin
  • 30,334
  • 10
  • 78
  • 137
xis
  • 24,330
  • 9
  • 43
  • 59
  • What kind of object is `results`? Ordinary list? Something else? – Kevin Aug 14 '14 at 17:59
  • 1
    See e.g. http://stackoverflow.com/q/12423614/3001761 – jonrsharpe Aug 14 '14 at 18:00
  • @Kevin e.g. results = [{key: value}, {key: value} ...] – xis Aug 14 '14 at 18:14
  • Ok. In that case, I agree with chepner that you don't need to worry about the warning here. – Kevin Aug 14 '14 at 18:19
  • The term `closure` which can be conceptually tricky made me overthink this error, which pylint has started throwing against a lot of my code. The example from the docs of code that throws this error makes it clear that this is a simple check against a basic error: https://pylint.pycqa.org/en/latest/user_guide/messages/warning/cell-var-from-loop.html#:~:text=Cell%20variable%20%25s%20defined%20in,for%20the%20closed-over%20variable. I'm disabling it in my pylintrc going forward. – Ben Quigley Jul 01 '22 at 14:51

2 Answers2

135

The name sort_key in the body of the lambda will be looked up when the function is actually called, so it will see the value sort_key had most recently. Since you are calling sort immediately, the value of sort_key will not change before the resulting function object is used, so you can safely ignore the warning. To silence it, you can make sort_key the default value of a parameter to the lambda:

results.sort(key=lambda k, sk=sort_key: get_from_dot_path(k, sk),
             reverse=(order == -1))
drew
  • 473
  • 5
  • 11
chepner
  • 497,756
  • 71
  • 530
  • 681
  • 13
    I'd err on the side of fixing the issue instead of ignoring the warning. If possible, I'd use `key=partial(get_from_dot_path, foo=sort_key)` instead of the lambda expression (assuming there is some parameter name `foo` defined by `get_from_dot_path` that you can use for a keyword argument; `partial` only allows filling in positional parameters exclusively from the left). – chepner Dec 13 '18 at 15:17
  • 1
    Ah I didn't realise this would fix it, I thought they were equivalent; in that case I agree. – Tim Diels Dec 14 '18 at 14:42
  • 3
    be aware that currently the trick does not always work https://github.com/PyCQA/pylint/issues/3107 – Daniel Pinyol Sep 16 '19 at 13:39
  • 1
    Using [`nonlocal`](https://docs.python.org/3/reference/simple_stmts.html#nonlocal) did _not_ help, I had to use [partial()](https://docs.python.org/3/library/functools.html#functools.partial) as suggested by @chepner. – Jens Feb 16 '21 at 10:49
  • Thanks. I had similar issue ``filtered_events = list(filter(lambda x, _phase_no=phase_no: 'phase_no' in x and x['phase_no'] == _phase_no, events))``, here the phase_no was causing problem for me – Dinesh P B Jun 15 '23 at 17:17
  • This post is now linked to from [Pylint docs](https://pylint.readthedocs.io/en/latest/user_guide/messages/warning/cell-var-from-loop.html) :) But I don't think the `partial` trick can be used if `k` is not directly used. For example, `get_from_dot_path(2*k, sort_key)` can't be translated to `partial` directly. – Abhijit Sarkar Jul 07 '23 at 02:14
11

Use functools.partial():

import functools
results.sort(key=functools.partial(get_from_dot_path, foo=sort_key),
             reverse=(order == -1))
mejdev
  • 3,509
  • 4
  • 24
  • 38
  • 1
    can confirm this not only gets rid of the warning, but really fixes the problem. Try for example making a list of lambdas print the range 1,10 - with lambdas, they all print 9, with functools.partial they print each value – TamaMcGlinn Jun 21 '21 at 12:11