Strange module reload behavior in spyder or IPython

Question

I have an issue re-running a script in spyder, which dynamically tabulates some of its own attributes.

Here is a minimal example that is representative of what I am doing. I have a source script that I run with the normal Run (F5) command. It runs in the same directory that it lives in:

runfile('C:/some/path/test.py', wdir='C:/some/path')

test.py

import sys

def x():
    pass

def y():
    pass

x.add = y.add = True

if __name__ == '__main__':
    a = [obj for obj in tuple(sys.modules[__name__].__dict__.values())
                if getattr(obj, 'add', False)]
    print(a)

I can re-run this script a couple of times, and always get the same expected result:

[<function x at 0x0000025E793DBD90>, <function y at 0x0000025E793DB598>]

I would expect that if I change the name of one of the functions, say from x to f, and the attribute assignment to f.add = y.add = True, I would get the same result, but with x changed to f. Instead, I get

[<function x at 0x0000025E793DB510>, <function y at 0x0000025E793DBBF8>, <function f at 0x0000025E793DBA60>]

Given that the new function f shows up in the list, but the old x is still there as well, I think that the module cache for the script is not being cleared properly.

I have been manually clearing the workspace using the eraser button over the console, but I feel like there must be a better solution, or at least an explanation of what is happening. Perhaps it is an issue with UMR?

I am running Spyder 3.3.3 with the following: Python 3.7.3 64-bit | Qt 5.9.6 | PyQt5 5.9.2 | Windows 10, IPython 7.4.9

Along [these lines](https://stackoverflow.com/questions/47738537/builtin-function-not-working-with-spyder/47738834#47738834) I expect — roganjosh, Sep 19 '19 at 19:47
Spyder caches variables and results without clearing them. That is not on accident. There is a way around it: [here](https://stackoverflow.com/a/48038888/10366273) — MyNameIsCaleb, Sep 19 '19 at 19:48
It has some cool uses. You can run a function that takes 10 mins to process data and return it to a global name, then just hash that function call out for every subsequent run and never have to incur the processing time again (until you wipe the namespace, that is). Until you learn that that name is just gonna hang around indefinitely, though, it's more likely that you will be bitten by it — roganjosh, Sep 19 '19 at 19:51
@MyNameIsCaleb. I would accept an answer with a combination of that and roganjosh's comment/GitHub thread — Mad Physicist, Sep 19 '19 at 19:54
@roganjosh. Makes sense. That's pretty much the explanation I was looking for. I didn't realize that the contaimination was two-way, which actually is pretty helpful. The option to clear vars before running is helpful — Mad Physicist, Sep 19 '19 at 19:56
My answer I linked to does point to the same solution as Caleb gave, but since I'm not too fussed about fleshing this out into an answer and you already invited them to give an answer, I will give them chance to put one together before I answer :) — roganjosh, Sep 19 '19 at 19:59
One thing I will look into, though, is whether it still applies for V4 because I have the Beta installed here and I think they may have changed the default behaviour — roganjosh, Sep 19 '19 at 20:00
@roganjosh. I think part of the issue is that I was expecting an intermediate option: not clearing the console variables, but completely reloading the module I care about — Mad Physicist, Sep 19 '19 at 20:02
A totally reasonable expectation. I'm _not sure_ whether the way they configured the global namespace allows it to flow one way even going forward, but it's totally what I expected to happen. I'll have a play. Separately, it looks like the "clear namespace on each script" is still off by default in V4 Beta — roganjosh, Sep 19 '19 at 20:05
I added it to the bottom but you can do `reset_selective` to clear specific ones matching regex — MyNameIsCaleb, Sep 19 '19 at 20:06
@MyNameIsCaleb. Yep, that's a pretty helpful option to. I think I'll just switch to `%run` line magic for now, since it does what I expect — Mad Physicist, Sep 19 '19 at 20:07

MyNameIsCaleb · Accepted Answer · 2019-09-19T20:31:24.633

This is a feature in Spyder that is built on purpose to allow you to continue running from stop points, especially when considering longer run times to generate specific results, and also powers the variable explorer function.

There is a way around it though following the instructions in this answer to Clear all variables before each run or by restarting.

However, as pointed out by roganjosh, clearing the namespace is not necessarily better.

You can run a function that takes 10 mins to process data and return it to a global name, then just hash that function call out for every subsequent run and never have to incur the processing time again (until you wipe the namespace, that is).

If you want to only reset one or a group of variables, you can use reset_selective

%reset_selective [-f] regex

@roganjosh no problem! – MyNameIsCaleb Sep 19 '19 at 20:31 — MyNameIsCaleb, Sep 19 '19 at 20:31

Strange module reload behavior in spyder or IPython

1 Answers1

Linked