3

I am trying to calculate the length of a list. When I run it on cmd, I get:

RuntimeError: maximum recursion depth exceeded in comparison

I don't think there's anything wrong with my code:

def len_recursive(list):
    if list == []:
        return 0
    else:
        return 1 + len_recursive(list[1:])
arshajii
  • 127,459
  • 24
  • 238
  • 287
Malyk
  • 173
  • 3
  • 15
  • 7
    How long a list are you passing in? Also, don't name a variable `list`; it masks the built-in type. – Martijn Pieters Jan 14 '13 at 20:55
  • 6
    I'm sure it's obvious, but in case someone comes across this in the wrong context, this is a really bad way to calculate the length of a list. Use `len()`. (I'm presuming that the OP is doing this for learning purposes). – Gareth Latty Jan 14 '13 at 20:55
  • You still haven't answered a very important question: how long is the list your passing to the function? You can use `len(l)` to determine it. – arshajii Jan 14 '13 at 21:06
  • @A.R.S. : for now, i am using a list of 200 elements. Since it's a homework i cannot use len(l) – Malyk Jan 14 '13 at 21:17
  • @Malyk: It's not working with 200? I believe the default recursion limit is at least 1000 on every implementation out there, and your code is obviously only doing 200 recursive calls, so it should work. (Unless it's already 800 calls deep in the call stack for some other reason?) – abarnert Jan 14 '13 at 21:27
  • @Malyk: Also, does the assignment actually say to do it recursively, or just to implement `len` however you want (without doing the obvious, `my_len = len`, `def my_len(l): return sum(1 for _ in l)`, etc.)? – abarnert Jan 14 '13 at 21:31
  • @abarnert: It just states not to use the `len`, so i thought doing it recursively was the easiest and fastest way – Malyk Jan 14 '13 at 21:37
  • The time complexity of an recursive function with the_list[1:] is O(n^2). The complexity of any iteration over list items is O(n). The complexity of `len()` is O(1). – hynekcer Jan 14 '13 at 21:38
  • @hynekcer: That's misleading. The time complexity of a recursive function is `O(n)`. It's only because this function slices the `list`, which is itself a linear operator in Python, that the whole thing ends up quadratic. If you used a type that can slice in constant type (and even a mutable type could do that with COW), it would be linear. – abarnert Jan 14 '13 at 21:56
  • @abarnert: The time necessary for copying the_list[1:] is proportional to `len(the_list) - 1` and the time necessary for all repeated copying is proportional to `sum(range(len(the_list)))` that is `n * (n - 1) / 2`. Also memory requirements are proportional to ˙n^2˙. – hynekcer Jan 14 '13 at 22:06
  • @hynekcer: Are you responding to what I actually wrote, or just the first two words? Read the sentence beginning with "It's only because…". – abarnert Jan 14 '13 at 22:11
  • @abarnert: I appreciate that your solution is O(n) and relative fast in Python3 if syntax is adapted. Yes. It is better to start not by denouncing as disaggreement can be fast and strong. COW for list slicing is related to Python 3 not to 2 (2.7) and so that comment could be inaccurate but not completely misleading. – hynekcer Jan 14 '13 at 23:19
  • @hynekcer: I wasn't trying to say that you were wrong, just that the way it was put could be misleading. I apologize if I didn't make that clear. It's not the recursion itself that makes it quadratic, but the fact that `lst[1:]` is linear instead of constant. Since someone trying to write code this way may be coming from a language where `rest` or `cdr` (or even `list::splice(it)`) is constant-time, it's an important distinction. – abarnert Jan 14 '13 at 23:46

6 Answers6

3

Don't use recursion unless you can predict that it is not too deep. Python has quite small limit on recursion depth.

If you insist on recursion, the efficient way is:

def len_recursive(lst):
    if not lst:
        return 0
    return 1 + len_recursive(lst[1::2]) + len_recursive(lst[2::2])
zch
  • 14,931
  • 2
  • 41
  • 49
  • 1
    You can get the current recursion depth limit with [`sys.getrecursionlimit()`](http://docs.python.org/2/library/sys.html#sys.getrecursionlimit) – Mark Ransom Jan 14 '13 at 21:02
  • Also, the OP's solution is likely to be much faster for small `n`, and not significantly slower until you get to `n` values way beyond the recursion limit, so I'm not sure "efficient" is really appropriate here. – abarnert Jan 14 '13 at 21:54
  • @abarnert, but it allows to go beyond the recursion limit. It only need `O(lg n)` stack depth. – zch Jan 14 '13 at 21:55
  • Yeah, the fixed version is fixed. And you're right that it can (at least theoretically) go up to nearly 2**1000 instead of just 1000, but that doesn't make it more _efficient_; it makes it more _correct_. See http://pastebin.com/xiPzh0vK; in 64-bit Apple Python 2.7.2, the OP's version is faster than yours in every range except 550-1350—except, of course, that at 1350 it's _failing_ a lot faster than yours is succeeding. Your code is clearly better; it's just about the word you use to describe _why_ it's better. – abarnert Jan 14 '13 at 22:09
2

The recursion depth in Python is limited, but can be increased as shown in this post. If Python had support for the tail call optimization, this solution would work for arbitrary-length lists:

def len_recursive(lst):
    def loop(lst, acc):
        if not lst:
            return acc
        return loop(lst[1:], acc + 1)
    return loop(lst, 0)

But as it is, you will have to use shorter lists and/or increase the maximum recursion depth allowed.

Of course, no one would use this implementation in real life (instead using the len() built-in function), I'm guessing this is an academic example of recursion, but even so the best approach here would be to use iteration, as shown in @poke's answer.

Community
  • 1
  • 1
Óscar López
  • 232,561
  • 37
  • 312
  • 386
  • thanks oscar. It's an assignment and i am not to use len() function. your version is good but as you mentioned the iteration approach, i will also do an iterative version – Malyk Jan 14 '13 at 21:26
2

As others have explained, there are two problems with your function:

  1. It's not tail-recursive, so it can only handle lists as long as sys.getrecursionlimit.
  2. Even if it were tail-recursive, Python doesn't do tail recursion optimization.

The first is easy to solve. For example, see Óscar López's answer.

The second is hard to solve, but not impossible. One approach is to use coroutines (built on generators) instead of subroutines. Another is to not actually call the function recursively, but instead return a function with the recursive result, and use a driver that applies the results. See Tail Recursion in Python by Paul Butler for an example of how to implement the latter, but here's what it would look like in your case.

Start with Paul Butler's tail_rec function:

def tail_rec(fun):
    def tail(fun):
        a = fun
        while callable(a):
            a = a()
        return a
    return (lambda x: tail(fun(x)))

This doesn't work as a decorator for his case, because he has two mutually-recursive functions. But in your case, that's not an issue. So, using Óscar López's's version:

@tail_rec
def tail_len(lst):
    def loop(lst, acc):
        if not lst:
            return acc
        return lambda: loop(lst[1:], acc + 1)
    return lambda: loop(lst, 0)

And now:

>>> print tail_len(range(10000))
10000

Tada.

If you actually wanted to use this, you might want to make tail_rec into a nicer decorator:

def tail_rec(fun):
    def tail(fun):
        a = fun
        while callable(a):
            a = a()
        return a
    return functools.update_wrapper(lambda x: tail(fun(x)), fun)
abarnert
  • 354,177
  • 51
  • 601
  • 671
1

Your exception message means that your method is called recursively too often, so it’s likely that your list is just too long to count the elements recursively like that. You could do it simply using a iterative solution though:

def len_iterative(lst):
    length = 0
    while lst:
        length += 1
        lst = lst[1:]
    return length

Note that this will very likely still be a terrible solution as lst[1:] will keep creating copies of the list. So you will end up with len(lst) + 1 list instances (with lengths 0 to len(lst)). It is probably the best idea to just use the built-in len directly, but I guess it was an assignment.

poke
  • 369,085
  • 72
  • 557
  • 602
1

Imagine you're running this using a stack of paper. You want to count how many sheets you have. If someone gives you 10 sheets you take the first sheet, put it down on the table and grab the next sheet, placing it next to the first sheet. You do this 10 times and your desk is pretty full, but you've set out each sheet. You then start to count every page, recycling it as you count it up, 0 + 1 + 1 + ... => 10. This isn't the best way to count pages, but it mirrors the recursive approach and python's implementaion.

This works for small numbers of pages. Now imagine someone gives you 10000 sheets. Pretty soon there is no room on your desk to set out each page. This is essentially what the error message is telling you.

The Maximum Recursion depth is "how many sheets" can the table hold. Each time you call python needs to keep the "1 + result of recursive call" around so that when all the pages have been laid out it can come back and count them up. Unfortunately you're running out of space before the final counting-up occurs.

If you want to do this recursively to learn, since you're want to use len() in any reasonable situation, just use small lists, 25 should be fine.

Some systems could handle this for large lists if they support tail calls

Paul Rubel
  • 26,632
  • 7
  • 60
  • 80
0

Python isn't optimising tail recursion calls, so using such recursive algorythms isn't a good idea. You can tweak stack with sys.setrecursionlimit(), but it's still not a good idea.

cleg
  • 4,862
  • 5
  • 35
  • 52