1

I'm watching a tutorial on python and it uses this example:

def add_spam(menu=[]):
    menu.append("spam")
    return menu

If you call add_spam() repeatedly then the menu list increases in size. This is new to me since I am from a C# background. But OK, fair enough.

To get around this it says that you should set the menu parameter to None but I cannot understand why that works. Here is the code they use:

def add_spam(menu=None):
    if menu is None:
        menu = []
    menu.append('spam')
    return menu

If you call it the first time though menu will be set to [], and the second time surely if it's 'remembering' the parameter as in the first example, menu will be [] at that point and therefore it will just append spam to the list as in the first example.

The video neglects any explanation other than you should use an immutable type so I cannot understand how this works.

Edit cos I still don't get it:

What I'm seeing is that the function captures the variable, so it takes menu=[] and stores that secretly somewhere, imagine a private variable called _menu, so that if you call it again it doesn't re-evaluate it just continues to use _menu and thus it grows.

In the second example, I do not understand why it isn't simply taking menu=None and storing that secretly as _menu, so _menu = None and then when you call the 2nd function, it sets _menu=[] and it continues exactly as the first example and it grows.

The fact that None is immutable doesn't seem relevant to me, as you're not doing None=[] you're doing menu=[] so menu then stops being what it was before and becomes [] and you can modify it as you like.

Unless it's some hard-coded feature that if you put None it will not do the copying behaviour then I do not understand the difference between the two.

NibblyPig
  • 51,118
  • 72
  • 200
  • 356
  • 1
    Conceptually a duplicate of https://stackoverflow.com/q/366422/2564301 – the *why* is [mentioned in a comment](http://effbot.org/zone/default-values.htm) but unfortunately the link in that to the official documentation has been declared Out of Cheese. – Jongware Jul 22 '18 at 14:15
  • Almost a dupe of https://stackoverflow.com/questions/1132941/least-astonishment-and-the-mutable-default-argument. – Ilja Everilä Jul 22 '18 at 14:51
  • None of the duplicate questions satisfactorily explain why :S – NibblyPig Jul 22 '18 at 14:58
  • "so _menu = None and then when you call the 2nd function, it sets _menu=[] and it continues exactly as the first example and it grows" is a misunderstanding. The name is bound to a new list in each invocation. You might find this a good read: https://nedbatchelder.com/text/names1.html – Ilja Everilä Jul 22 '18 at 15:07
  • But we just established that it's not bound to a new list in each invocation though, as it uses the same variable just stored away. That's the part I don't get. – NibblyPig Jul 22 '18 at 15:13
  • 1
    It uses the variable that's stored away for the default value of the parameter. Once the name is rebound to a new object, that value doesn't matter any more – JoshuaF Jul 22 '18 at 15:35
  • 1
    Objects created within the body of a function (as opposed to the definition) are certainly created anew each call. – JoshuaF Jul 22 '18 at 15:36
  • You really should read the "Python has names" article. I get the feeling that you think of assignment as modifying objects somehow, when it simply rebinds names. – Ilja Everilä Jul 22 '18 at 15:46
  • Yes, I think it's just that it makes no sense to have this behaviour that is confusing me. I think I get it now, thanks. From what I can see if you provide a default value that is mutable such as a list, then it will just ruin your day because `menu=[]` doesn't mean the default for menu will be [], it means the default for menu could be any possible list based on the previous calls and you probably shouldn't be using it. – NibblyPig Jul 22 '18 at 15:49
  • There are times when it's useful. When subsequent calls to a function might involve recomputing a lot of the same information, storing that information in a mutable default argument is a very clean way to save a lot of effort. – JoshuaF Jul 22 '18 at 16:18
  • 1
    But you can only ever do it once, safely, and if anything goes wrong in your program it can be left in a broken state. And if anyone externally calls your function they will break it and/or receive unexpected results. If it's computationally expensive, safer to just handle that yourself, no? – NibblyPig Jul 22 '18 at 17:10
  • The intended use case is, I believe, something like a dynamic program, which passes all the previously computed results between recursive calls with a parameter intended only for internal use within the function. Allowing subsequent calls of the function to use those previously computed results might save a lot of time (although, like you said, the programmer could provide for that explicitly themselves). – JoshuaF Jul 23 '18 at 12:25
  • http://effbot.org/zone/default-values.htm – JoshuaF Jul 23 '18 at 12:28

2 Answers2

4

Python’s default arguments are evaluated once when the function is defined, not each time the function is called (like it is in say, Ruby).

https://docs.python-guide.org/writing/gotchas/

From Python's docs, some more detail:

Default parameter values are evaluated from left to right when the function definition is executed. This means that the expression is evaluated once, when the function is defined, and that the same “pre-computed” value is used for each call.

https://docs.python.org/3/reference/compound_stmts.html#function-definitions

Lists are mutable objects, whereas None is immutable, hence the behavior you see.

Pedro Silva
  • 4,672
  • 1
  • 19
  • 31
  • Can you expand on how this works out, using OP's code as an example? – Jongware Jul 22 '18 at 14:23
  • If the same pre-computed value is used for each call, why does calling the first example multiple times cause `menu` to get bigger and bigger? Surely it would use the precomputed value of `[]` every time? And if it doesn't do this, then the second example wouldn't fix it either. – NibblyPig Jul 22 '18 at 14:23
  • But the reference to None is gone, when it gets set to [], so all you have then is `menu` set to `[]` so the immutability doesn't seem like it should be a factor – NibblyPig Jul 22 '18 at 14:28
  • Sure, if you nest calls like so: `add_spam(add_spam())` (ie, if you reuse the returned value). But if you call `add_spam` with no arguments repeatedly, then menu gets bound to `None` repeatedly, and "overwritten", but *only* inside the scope of the function. – Pedro Silva Jul 22 '18 at 14:33
  • Still going around in circles though, why does it get bound to None repeatedly when it says `the expression is evaluated once, when the function is defined, and that the same “pre-computed” value is used for each call` and yet menu doesn't get set to [] repeatedly in the 1st example – NibblyPig Jul 22 '18 at 14:35
  • *Because you're not passing an argument to the function!* Hence, the default parameter kicking in. A default parameter is *evaluated* at function definition time, but parameter *binding* happens at function call time. – Pedro Silva Jul 22 '18 at 14:38
  • That doesn't really clear anything up in my head. I still can't understand why one example works and one doesn't. You don't pass anything to example 1, and it persists modifications to menu, you don't pass anything to example 2, and it *doesn't* persist modifications to menu, but I can't comprehend why not – NibblyPig Jul 22 '18 at 14:40
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/176516/discussion-between-pedro-silva-and-nibblypig). – Pedro Silva Jul 22 '18 at 14:41
3

Setting menu = [] inside the function is just rebinding the name menu to a new object inside the scope of that function call. It doesn't change the original menu object in the scope where the function was defined. Appending to menu within the function, on the other hand, actually modifies the object in memory associated with the name menu in the function definition (because lists are mutable). This object was created when the function was first defined, and is shared between calls of the function.

If you want to understand more, look into the python scoping/namespace behavior, of which default argument sharing is mostly just a subset. But basically,

def foo(bar=None): #executed once
    bar = [] #executed every time the function is called
JoshuaF
  • 1,124
  • 2
  • 9
  • 23
  • 2
    Yes, I think I sort of get it now. In the first instance it has a hidden variable (I read about __defaults__ but let's keep it simple) called say, _menu. So it does `_menu = []` and when you call it a 2nd time it does `menu = _menu` to provide your value, ie. retrieves the stored value so everything is appended. In the second instance, it does `_menu = None` the first time, and when you call it a second time, it does menu = _menu so both will be `None`, and when you set `menu = []` it doesn't change `_menu` since `_menu` does not point to `menu`. – NibblyPig Jul 22 '18 at 15:30