Python: delete all variables except one for loops without contaminations

Question

%reset
%reset -f

and

%reset_selective a
%reset_selective -f a

are usefull Python alternative to the Matlab command "clear all", in which "-f" means "force without asking for confirmation" and "_selective" could be used in conjunction with

who_ls

to selectively delete variables in workspace as clearly shown here https://ipython.org/ipython-doc/3/interactive/magics.html .

Now I am managing loops in which I am going to define a large number of variables, for example

for j in range(1000):
    a = crazy_function1()
    b = crazy_function2()
    ...
    m = crazy_function18()
    n = crazy_function19()
    ...
    z = crazy_functionN()

and at the end of each cycle I want to delete ALL variables EXCEPT the standard variables of the Python workspace and some of the variables I introduced (in this example only m and n). This would avoid contaminations and memory burdening hence it will make the code more efficient and safe.

I saw that "who_ls" result looks like a list, hence I thought at a loop that delete all variables that are not equal to m or n

for j in range(1000):
    a = crazy_function1()
    b = crazy_function2()
    ...
    m = crazy_function18()
    n = crazy_function19()
    ...
    z = crazy_functionN()
    if who_ls[j] != m or who_ls[j] != n:
         %reset_selective -f who_ls[j]

but it doesn't work as who_ls looks as a list but it doesn't work as a list. How would you modify the last lines of code? Is there anything like

 %reset_selective -f, except variables(m, n)

?

Answer 1

The normal approach to limiting the scope of variables is to use them in a function. When the function is done, its locals disappear.

In [71]: def foo():
    ...:     a=1
    ...:     b=2
    ...:     c=[1,2,3]
    ...:     d=np.arange(12)
    ...:     print(locals())
    ...:     del(a,b,c)
    ...:     print(locals())
    ...:     
In [72]: foo()
{'d': array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11]), 'c': [1, 2, 3], 'a': 1, 'b': 2}
{'d': array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11])}

==================

%who_ls returns a list, and can be used on the RHS, as in

xx = %who_ls

and then that list can be iterated. But note that this is a list of variable names, not the variables themselves.

for x in xx: 
    if len(x)==1:
        print(x)
        # del(x)  does not work

shows all names of length 1.

======================

A simple way to use %reset_selective is to give the temporary variables a distinctive name, such as a prefix that regex can easily find. For example

In [198]: temp_a, temp_b, temp_c, x, y = 1,'one string',np.arange(10), 10, [1,23]
In [199]: who_ls
Out[199]: ['np', 'temp_a', 'temp_b', 'temp_c', 'x', 'y']
In [200]: %reset_selective -f temp 
In [201]: who_ls
Out[201]: ['np', 'x', 'y']

====================

Here's an example of doing this deletion from a list of names. Keep in mind that there is a difference between the actual variable that we are trying to delete, and its name.

Make some variables, and list of names to delete

In [221]: temp_a, temp_b, temp_c, x, y = 1,'one string',np.arange(10), 10, [1,23]
In [222]: dellist=['temp_a', 'temp_c','x']

Get the shell, and the user_ns. who_ls uses the keys from self.shell.user_ns.

In [223]: ip=get_ipython()
In [224]: user_ns=ip.user_ns

In [225]: %who_ls
Out[225]: ['dellist', 'i', 'ip', 'np', 'temp_a', 'temp_b', 'temp_c', 'user_ns', 'x', 'y']

In [226]: for i in dellist:
     ...:     del(user_ns[i])
     ...:     
In [227]: %who_ls
Out[227]: ['dellist', 'i', 'ip', 'np', 'temp_b', 'user_ns', 'y']

So we have to look up the names in the user_ns dictionary in order to delete them. Note that this deletion code creates some variables, dellist, i, ip, user_ns.

==============

How many variables are you worried about? How big are they? Scalars, lists, numpy arrays. A dozen or so scalars that can be named with letters don't take up much memory. And if there's any pattern in the generation of the variables, it may make more sense to collect them in a list or dictionary, rather than trying to give each a unique name.

In general it is better to use functions to limit the scope of variables, rather than use del() or %reset. Occasionally if dealing with very large arrays, the kind that take a meg of memory and can create memory errors, I may use del or just a=None to remove them. But ordinary variables don't need special attention (not even in an ipython session that hangs around for several days).

Answer 2

I was looking help for the same task and I noticed the comment from hpaulj, about that %who_ls return a list of names then I remembered a little about metaprogrammin in R with the command eval(parser(text = text)), then I looked for and equivalent of the previous command in python to delete all the variables in lst_arch using the names of the variables (R's eval(parse(text=text)) equivalent in Python)

    a = 1
    b = 1

    lst_arch = %who_ls
    lst_arch.append('lst_arch')
    lst_arch.append('x')

    for x in lst_arch:
        exec(f'del(' + x + ')')

If we want to filter certain variables from lst_arch, we can use regular expression or something like that to delete just those variables we want to delete.