8

This answer explains how to create test cases dynamically.

The answer's code:

class Tests(unittest.TestCase):
    def check(self, i, j):
        self.assertNotEquals(0, i-j)

for i in xrange(1, 4):
    for j in xrange(2, 6):
        def ch(i, j):
            return lambda self: self.check(i, j)
        setattr(Tests, "test_%r_%r" % (i, j), ch(i, j))

I've tested and it works, but I can't just figure out how?

I have trouble understanding the lambda self: magic in play here, mainly:

  1. Is the lambda used here to perform the exact opposite of functools.partial() (i.e. to create a wrapper function with one extra parameter that is not yet known)
  2. Is self a meaningful keyword or would lambda spam would work just as well?
  3. What point is that lambda evaluated?
  4. How come the .check() is perfectly fine outside the classes scope?
  5. How do you do this without lambda? - if I understand correctly, you should be able to do without lambda (not to mention that I agree with Guido and Alex that it is an eyesore and I want to do without :)
Community
  • 1
  • 1
Kimvais
  • 38,306
  • 16
  • 108
  • 142

3 Answers3

5

First of all: The lambda is returned from a function to prevent that modified values of i and j will be used later.

Compare to the following:

for i in xrange(1, 4):
    for j in xrange(2, 6):
        setattr(Tests, "test_%r_%r" % (i, j), lambda self: self.check(i, j))

This will not work as expected, as all the lambdas will share the same namespace and will use the values of i and j after the end of the loop. If we use a separate function call, we introduce a new closure every time that captures the values of i and j at the point in time where the function is called.

We could achieve the same by saving the current values of i and j as default arguments of the lambda. For good measure, let's also use itertools.product to avoid the nested loop:

from itertools import product
from operator import methodcaller

for i, j in product(range(1, 4), range(2, 6)):
  setattr(Tests, "test_%r_%r" % (i, j),
          lambda self, i=i, j=j: self.check(i, j))

Is the lambda used here to perform the exact opposite of functools.partial() (i.e. to create a wrapper function with one extra parameter that is not yet known)

Not really. It just calls the check(i, j) method on whatever it is given as an argument. This is used here to dynamically add methods to the Tests class.

Is self a meaningful keyword or would lambda spam would work just as well?

spam would work just as well. They choose self here due to convention because the lambda represents a method.

What point is that lambda evaluated?

As soon as test_[i]_[j]() is called on an instance of Tests.

How come the .check() is perfectly fine outside the classes scope?

Because it's inside a lambda will only be called later with an instance of Tests as the self argument.

BenMorel
  • 34,448
  • 50
  • 182
  • 322
Niklas B.
  • 92,950
  • 18
  • 194
  • 224
  • 1
    ok "Not really. It just calls the check(i, j) method on whatever it is given as an argument. This is used here to dynamically add methods to the Tests class." was the thing that I did not realize. Now that you mention it, it is self-evident (pun intended) – Kimvais Apr 27 '12 at 15:11
  • The methodcaller modification (which would make the code more readable too, IMO) gives me `TypeError: methodcaller expected 1 arguments, got 0` – Kimvais Apr 27 '12 at 15:18
  • @Kimvais: This is very strange. It seems like the function is not bound correctly when assigned to the class attribute.. This might be due to the fact that it's not a real lambda, but a function defined on the C side of Python. I guess I will open a separate question for that. – Niklas B. Apr 27 '12 at 15:28
  • @Kimvais: Meanwhile, you can also use the default argument trick. I edited my answer. – Niklas B. Apr 27 '12 at 15:30
  • Another way would be to use define a `make_func(f)` function that just returns a real function: `return lambda *args, **kw: f(*args, **kw)`. Then we can use `methodcaller` like follows: `make_func(methodcaller("check", 1, 2))`. – Niklas B. Apr 27 '12 at 15:38
  • Please post the link to the question you will be opening here. – Kimvais Apr 27 '12 at 15:51
  • @Kimvais: Actually I've already answered the question myself using a bit of common sense. What happens is that if you assign a function to a class attribute, it becomes an unbound method. As soon as an instance is created from the class, all the unbound methods are bound to the object instance (that is, they are partially applied to the new `self`). Now if we assign something to a class attribute that is not a function, but only callable, Python doesn't know that we want it to become a method, so it doesn't convert it to an unbound method. – Niklas B. Apr 27 '12 at 16:18
5

a_function = lambda x: do_something_with(x)

is equivalent to

def a_function(x):
    return do_something_with(x)

In your particular example, the functions created are instance methods. Thus for example you could write:

class Tests:
...
   test_1_2 = lambda self: self.check(1, 2)

which would be equivalent to:

class Tests:
...
   def test_1_2(self):
       return self.check(1, 2)

Now, since the identifiers are generated dynamically, hence use of setattr.


  1. Is the lambda used here to perform the exact opposite of functools.partial() (i.e. to create a wrapper function with one extra parameter that is not yet known)

No, not really. In fact partial could be used here as well.

test_1_2 = lambda self: self.check(1, 2)

would became

test_1_2 = partial(Tests.check, i=1, j=2)
  1. Is self a meaningful keyword or would lambda spam would work just as well?

Instance method's first argument always needs to be the instance. The convention is to name that parameter self, but you can use any other name. I would not advise it, as it will make your code harder to read and understand.

  1. What point is that lambda evaluated?

lambda creates anonymous function, which as any function is evaluated upon call.

  1. How come the .check() is perfectly fine outside the classes scope?

It isn't. It's self.check(), where self is an instance of Tests class.

BenMorel
  • 34,448
  • 50
  • 182
  • 322
vartec
  • 131,205
  • 36
  • 218
  • 244
  • 2
    Actually, `operator.methodcaller("check", 1, 2)` is preferrable over `partial` here because it uses duck typing and does not require an instance of `Tests`. This comment also applies to your answer to the other question. – Niklas B. Apr 27 '12 at 14:38
  • @NiklasB.: ok, but the question was if `partial` is something opposite of `lambda`, not if that's a best way – vartec Apr 27 '12 at 15:20
  • Sure, maybe I should have better added it to your other answer. – Niklas B. Apr 27 '12 at 15:22
  • By the way, I just checked it and it seems like the `Tests.method = partial(Tests.check, i=1, j=2)` trick doesn't work (but neither does `Tests.method = operator.methodcaller("check", 1, 2)`). I suspect that the method is not bound correctly because it's not a real function but only a callable, but I'm not sure what's going on. – Niklas B. Apr 27 '12 at 15:32
3

lambda returns a function. self is its argument, and i and j are enclosed inside of it

So, ch(i, j) is a function of i and j which returns a function of self.

self does not have any magical meaning here as nothing in Python - every variable and function are explicit except of some built-ins. But as soon as these methods are attached to class, self becomes their 1st argument.

This means

  • after the loop The Tests class gets multiple methods called check_i_j

  • each of them has values i and j enclosed in it

  • each of them has their 1st param called self which is good cause they are defined inside of class, they are instance methods, and thus their 1st parameter should be called self by convention

Guard
  • 6,816
  • 4
  • 38
  • 58
  • BTW, I thing the code could be much simplified by just writing setattr(Tests, "test_%r_%r" % (i, j), lambda self: self.check(i, j)) – Guard Apr 27 '12 at 13:32
  • No, that does something completely different. – Niklas B. Apr 27 '12 at 13:33
  • @NiklasB. please explain - what's the diference between defining the ch function and instantly calling it and just putting it's value. i and j are enclosed anyway, right? – Guard Apr 27 '12 at 13:36
  • 1
    All the lambdas will share the same closure, so they will all work with the values of `i` and `j` at the end of the loops. Check my answer for more details. The function call is used in the other answer to force the creating of a new closure for every lambda. – Niklas B. Apr 27 '12 at 13:36
  • I know such problem in JS, didn't expect it's applicable to Python. Though now I realize this is the same here – Guard Apr 27 '12 at 13:38
  • It's not a problem, you just have to be aware of it. – Niklas B. Apr 27 '12 at 13:39