Python Lambda Count/Loop Function

Question

I'm sorry if this is a question answered elsewhere. Searching through Google and Stackforum I didn't find anything from which I could extrapolate the answers; but I feel like part of that is me.

I'm trying to work out lambdas as a concept, and as part of that I'm kinda looking for ways to use it.

SO, if this is a colossally stupid thing to do with lambda from a function standpoint, feel free to let me know and explain. But either way, I still want to know the answer/still want to know how to do this with the python language.

So, for testing purposes I have:

my_test = 'test_name'
testlist = ['test_name', 'test_name_dup', 'test_name_dup_1', 'test_name_dup_3']

I'm looking to use lambda to create one function that loops through and returns the first test_name_# that isn't in the testlist. The functionality will eventually be applied to filenames, but for testing purposes I had to get away from actually reading the filenames--gave me too many more ways to mess something up.

But my_test has to be able to change, and the test list will be a list of filepaths.

So, I'm looking for a function like:

new_name = lambda x: my_test + '_' + str(x)

But the initial value should be x = 1, and it should continue until new_name is not in testlist. Seems like:

bool(new_name not in testlist)

might be something work with.

But I can't figure out a way to set the initial x to 1, and have it loop through with (x+1) until the bool is true.

I know this is possible as I've found some CRAZY lambda examples out there that are looping through lines in a file. I just couldn't quite make sense of them (and didn't have any way to play with them as they were dealing with things outside my programming level.

On a related note, could I add values to the beginning of this loop? (i.e. can I have it check for test_name, then test_name_dup, then test_name_dup_#)?

Thanks in advance for the help! Lambdas (while very cool) totally mess with my head.

Answer 1

Lambdas are just another way of defining a function

def foo(x):
    return x + x

is the same as

foo = lambda x: x + x

So let's start with a function to do what you want:

def first_missing(items, base):
    for number in itertools.count():
        text = base + '_' + str(number)
        if text not in items:
             return text

The first thing to note is that you can't use loops inside a lambda. So we'll need to rewrite this without a loop. Instead, we'll use recursion:

def first_missing(items, base, number = 0):
        text = base + '_' + str(number)
        if text not in items:
             return text
        else:
             return first_missing(items, base, number + 1)

Now, we also can't use an if/else block in a lambda. But we can use a ternary expression:

def first_missing(items, base, number = 0):
        text = base + '_' + str(number)
        return text if text not in items else first_missing(items, base, number + 1)

We can't have local variables in a lambda, so we'll use a trick, default arguments:

def first_missing(items, base, number = 0):
        def inner(text = base + '_' + str(number)):
            return text if text not in items else first_missing(items, base, number + 1)
        return inner()

At this point we can rewrite inner as a lambda:

def first_missing(items, base, number = 0):
        inner = lambda text = base + '_' + str(number): text if text not in items else first_missing(items, base, number + 1)
        return inner()

We can combine two lines to get rid of the inner local variable:

def first_missing(items, base, number = 0):
    return (lambda text = base + '_' + str(number): text if text not in items else first_missing(items, base, number + 1))()

And at long last, we can make the whole thing into a lambda:

first_missing = lambda: items, base, number = 0: (lambda text = base + '_' + str(number): text if text not in items else first_missing(items, base, number + 1))()

Hopefully that gives you some insight into what you can do. But don't ever do it because, as you can tell, lambdas can make your code really hard to read.

Answer 2

There's no need to use a lambda in this case, a simple for loop will do:

my_test  = 'test_name_dup'  
testlist = ['test_name', 'test_name_dup','test_name_dup_1', 'test_name_dup_3']

for i in xrange(1, len(testlist)):
    if my_test + '_' + str(i) not in testlist:
        break

print my_test + '_' + str(i)
> test_name_dup_2

If you really, really want to use a lambda for this problem, you'll also have to learn about itertools, iterators, filters, etc. I'm gonna build on thg435's answer, writing it in a more idiomatic fashion and explaining it:

import itertools as it

iterator = it.dropwhile(
    lambda n: '{0}_{1}'.format(my_test, n) in testlist,
    it.count(1))

print my_test + '_' + str(iterator.next())
> test_name_dup_2

The key to understanding the above solution lies in the dropwhile() procedure. It takes two parameters: a predicate and an iterable, and returns an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every element.

For the iterable, I'm passing count(1), an iterator that produces an infinite number of integers starting from 1.

Then dropwhile() starts to consume the integers until the predicate is false; this is a good opportunity for passing an in-line defined function - and here's our lambda. It receives each generated integer in turn, checking to see if the string test_name_dup_# is present in the list.

When the predicate returns false, dropwhile() returns and we can retrieve the value that made it stop by calling next() on it.

Answer 3

You can combine a lambda with itertools.dropwhile:

import itertools
n = itertools.dropwhile(lambda n: 'test_name_dup_%d' % n in testlist, range(1, len(testlist))).next()

As to your last question, you can write a generator for names, like:

def possible_names(prefix):
    yield prefix
    yield prefix + '_dup'
    n = 0
    while True:
        n += 1
        yield '%s_dup_%d' % (prefix, n)

and then use this generator with dropwhile:

unique_name = itertools.dropwhile(lambda x: x in testlist, possible_names('test_name')).next()
print unique_name

Answer 4

You are a bit off the track. Lambdas are nothing but "simple" functions, often used for their fast syntax in functional programming. They are the perfect companion built-in functions "map", "reduce", "filter" but also for more complicated functions defined into itertools. Therefore the most useful thing to do with them is to generate/manipulate iterable objects (especially lists). Note that lambdas will slow down your code in most of the cases if compared to list comprehensions/normal loops and will make it harder to be read. Here is an example of what you want to do with lambdas.

>>> filter(lambda i: i!=(0 if len(testlist[i].split("_"))==3 else int(testlist[i].split("_")[-1])), range(len(testlist)))[0]
2

Or you could use more complicated functions with itertools. Anyhow I strongly suggest you not to use lambdas for this kind of assignments, as the readability is terrible. I'd rather use a well structured for loop, which is also faster.

[Edit]

To prove that lambdas+builtins are not faster than list comprehensions: consider a simple problem, for x in range(1000) create a list of x shifted by 5.

$ python -m timeit 'map(lambda x: x>>5, range(1000))' 1000 loops, best of 3: 225 usec per loop

$ python -m timeit '[x>>5 for x in range(1000)]'10000 loops, best of 3: 99.1 usec per loop

You have a >100% performance increase without lambdas.

Answer 5

I prefer the list comprehension or iterator method. Makes for easy one liners that I feel are pretty easy to read and maintain. Quite frankly, lambdas belong some places, here I believe its less elegant a solution.

my_test = 'test_name'
prefix = 'test_name_dup_'
testlist = ['test_name','test_name_dup','test_name_dup_1','test_name_dup_3']

from itertools import count
print next('%s%d' % (prefix, i) for i in count(1) if '%s%d' % (prefix, i) not in testlist)

This returns the first not-found instance in the sequence, which I think is the cleanest.

Of course, if you prefer a list from a definite range, you can modify it to become a list comprehension:

print ['%s%d' % (prefix, i) for i in xrange(0,5) if '%s%d' % (prefix, i) not in testlist]

returns:

['test_name_dup_0', 'test_name_dup_2', 'test_name_dup_4']