Flask - default parameters ignored on recursive function call (values accumulated between separate calls)

Question

I have been running into an issue when calling a flask api in parallel (problem occurs with only 2 instances). In my full code, I request data from a table and then create a json tree that will then be displayed on the front-end. I noticed that my trees were looking strange, the first one was OK, but the second tree contained all the nodes of the first one and then only the nodes of the second tree).

I am calling a recursive function to build my tree and it looks like some of the variables were "shared" between the two parallel executions (which does not make sense to me, as I am calling a method of 2 different instances of an object).

I created the example below that replicates my issue.

The result variable should be a list containing a number for each recursive execution (so for 5, I would expect [1 2 3 4 5]

But when I execute the code several times (by doing refresh), I can see that my result list is growing much bigger (so after couple of refreshes, my list is [1 2 3 ...75] (and the recursion can only be executed between 5 and 15 times)

I do not specify the numbers parameter in my initial call of the recursive method:

result = test_recursive.recursive_function(random.randint(5, 15))

as I expect it to be initalized to an empty array as specified in the default value:

def recursive_function(self, iterations, numbers=[]):

The numbers list does not seem to be re-initalized to an empty array, but it's keeping the value of the last execution. I am a bit puzzled by how this is even possible...

I can fix my problem by explicitly specifying the numbers parameter result = test_recursive.recursive_function(random.randint(5, 15), [])

I am however not really happy about this solution, because I have many more parameters with default values in my real recursive function and I do not understand the root cause of this issue.

Is something wrong in my code or couéd this be a bug ?

flask blueprint

class get_dimension_json(Resource):
    def get(self, dimension, hierarchy):
        try:
            test_recursive = TestRecursive(dimension, random.randint(1, 5) / 10)
            result = test_recursive.recursive_function(random.randint(5, 15))
            log.debug('result %s %s : %s', dimension, hierarchy, result)
            return {'msg': 'result}, 200
        except Exception as e:
            log.debug('get_dimension_json get %s %s finished with errors', dimension, hierarchy)
            log.debug(e)
            return {'msg': repr(e)}, 500

api.add_resource(
    get_dimension_json,
    '/api/report/dimension-json/<string:dimension>/<string:hierarchy>',
)

testRecursive.py

import time
import logging

log = logging.getLogger(__name__)


class TestRecursive:
    def __init__(self, name, seconds_delay):
        self.name = name
        self.seconds_delay = seconds_delay

    def recursive_function(self, iterations, numbers=[]):
        time.sleep(self.seconds_delay)
        log.debug('%s - %s', self.name, iterations)
        numbers.append(len(numbers) + 1)
        if iterations > 0:
            self.recursive_function(iterations - 1, numbers)
        return numbers

score 1 · Accepted Answer · answered Jan 14 '21 at 10:05

1

That's the way python works / a common pitfall. The initialization value/object for numbers is computed only once when the function is first interpreted (on import / run module), this object is re-used on subsequent calls. This is why default arguments should always be immutable (e.g., tuple). For more information read about "mutable default arguments" in python.

answered Jan 14 '21 at 10:05

Yuri Feldman

2,354
1
20
23

1

Thank you, knowing that could have saved me a lot of time... I read about it and understand how to fix my code, but I don't really get the benefits of the way this has been implemented in python. Seems to me like only leads to code that is more difficult to test, as you would have to test each function with default parameters multiples times to ensure there are no unwanted side-effects between calls... Or just not use default parameters at all if they are mutable... – marcel f Jan 14 '21 at 10:22
There's a SO thread specifically on why it is that way (tl;dr;: because in python a function is an object, and it just stores the default values). There are also indications to how this can be useful. https://stackoverflow.com/questions/1132941/least-astonishment-and-the-mutable-default-argument – Yuri Feldman Jan 14 '21 at 13:53

Flask - default parameters ignored on recursive function call (values accumulated between separate calls)

1 Answers1