0

in order to automatically generate parameterized tests, I am trying to add methods to a class in by freezing some parameters of an existing method. Here is the piece of Python 3 code

class A:
    def f(self, n):
        print(n)

params = range(10)

for i in params:
    name = 'f{0}'.format(i)
    method = lambda self: A.f(self, i)
    setattr(A, name, method)

However, the following lines then produce rather disappointing output

a = A()
a.f0()

prints "9" (instead of "0"). I must be doing something wrong, but I can't see what. Can you help ?

Thanks a lot

Edit: this question is indeed a duplicate. I would like to acknowledge the quality of all comments, which go much deeper than the raw answer.

Sebastien
  • 246
  • 1
  • 12
  • 1
    for such things in tests try [mock](http://www.voidspace.org.uk/python/mock/) – warvariuc Feb 07 '14 at 16:33
  • @warwaruk: after looking at the quick start guide, I am not sure how mock can help... but I am quite happy to learn about this package. Might prove useful at some point. – Sebastien Feb 07 '14 at 17:21

2 Answers2

2

Try

method = lambda self, i=i: A.f(self, i)

because otherwise when you call the method i's value may have changed

warvariuc
  • 57,116
  • 41
  • 173
  • 227
  • Is this somehow connected to the fact that Python variables are somehow different from e.g. Java references? Where could I learn more about these subtleties? – Sebastien Feb 07 '14 at 16:38
  • 1
    This is related to variable scopes. `lambda` is a simple function with body of `A.f(self, i)`. When you call a method with this body, `i`'s value is requested. It is taken from outer scope. Search SO and google for closures – warvariuc Feb 07 '14 at 16:44
  • 1
    @Sebastien It's due to the fact that, without the `=i` part, the value of `i` will not be determined until the lambda function is executed (in which case it goes with the most recent value assigned to `i`, which after your loop will be `9`). This is true for all Python functions, in fact, and is the reason why you can have issues when referring to nonlocal variables (but is also the reason you can declare a function before declaration of the nonlocal value it uses). There's quite a bit more to scope in Python than that, but that's how it basically works. – JAB Feb 07 '14 at 16:45
  • 1
    @Sebastien Python variables are actually very similar to Java variables (there are some oddities around primitives not being objects, but nothing that affects common operations). It's languages like C, C++ which differ significantly from Python and Java. By the way, Java sidesteps the moral equivalent of this problem (with an inner class instead of a `lambda`/closure) by requiring `i` to be `final`, i.e. non-modifiable. –  Feb 07 '14 at 16:45
  • @delnan Java still requires them to be declared no later than compile time, while with Python the variable only has to be declared in a scope reachable by whatever references it before the referent executes. If it never executes, the variable doesn't even need to exist to begin with. (The reason `i=i` freezes the value is because it is only evaluated when the function is constructed rather than whenever it is called [which is why things like `def x(l=None):\n l or [] #...` are preferred to `def x(l=[]):\n l #...`; the latter won't have an empty list default if the list is appended to inside].) – JAB Feb 07 '14 at 16:53
  • Ok, I understand the keyword I was missing is scope. Have to admit scope is sometimes disturbing in Python. All these answers are very helpful (I like the mention of the final keyword in Java). – Sebastien Feb 07 '14 at 17:18
  • 1
    @Sebastien Scope isn't really the issue here, time of evaluation is. In fact, without this behavior, it would be a lot harder to write closures in Python. (Note that Python [3] actually defaults to `final`-like behavior for all nonlocal variables; you can modify attributes of the variables, just as you can with non-final members of `final` objects in Java, but you cannot assign a different object to the nonlocal variable without first declaring the variable as `nonlocal` inside the closure.) – JAB Feb 07 '14 at 17:35
  • 1
    There's also `global`, which acts similarly to `nonlocal` but for global/module-level variables. It also has the difference that, while the statement `nonlocal x` requires `x` to exist in lexical scope when the function is created, the statement `global x` does not (but if you don't create the variable at execution time of the module, all uses of that variable in functions will require the usage of `global x` even if you don't assign anything to it in those others; while this pattern does have some uses [I remember using it with `ply`/`llvmpy`], it's not something you'll want to do regularly). – JAB Feb 07 '14 at 17:44
  • @Sebastien, see [this](http://stackoverflow.com/questions/233673/lexical-closures-in-python) question – warvariuc Feb 08 '14 at 04:31
  • Actually, I was slightly off with my statement about _all_ uses requiring `global x`, but I don't have the source code at the moment to remind myself of why I ended up having `global x` in multiple places... I may have been doing lazy initialization in multiple functions, perhaps. – JAB Feb 10 '14 at 17:54
1

The best way to "freeze" parameters in Python is to use functools.partial. It's roughly equivalent to warwaruk's lambda version, but if you have a function with lots of arguments yet only want to freeze one or two of them (or if you only know certain arguments and don't care about the rest) using partial is more elegant as you only specify the arguments you want to freeze rather than having to repeat the whole function signature in the lambda.

An example for your program:

class A:
    def f(self, n):
        print(n)

from functools import partial

for i in range(10): # params
    setattr(A, 'f{0}'.format(i), partial(A.f, n=i))

Depending on which version of Python 3 you're using, you may not need to include the 0 in the string format placeholder; starting with 3.1, iirc, it should be automatically substituted.

JAB
  • 20,783
  • 6
  • 71
  • 80