1

I'm looking at a Python codebase's memory consumption. That codebase leverages pandas and numpy to manipulate huge data frames.

When we're done with intermediate data representation, we want to free it to release some memory. One of my colleagues noticed that there is a difference between calling del dataframe and del [[dataframe]]:

Screenshot of Jupyter notebook trying the different approaches

From that they concluded "if del df would do the same thing as del [[df]] the memory from cell 12 should be the same as the one from cell 10, As between 12 and 14, but you can see that not all the memory was freed to be used again."

I agree with their understanding of what happens in practice logic here. However, when trying to build a theoretical understanding to back that code up, I can't find a clear explanation of why this would be different.

Looking at this answer, the definition of del statements taking a target_list should make del my_list and del [my_list]/del [[my_list]]/del [[[my_list]]] equivalent:

del_stmt ::=  "del" target_list

And yet in this answer the author uses del [[df1, df2]] instead of del df1, df2 or even del [df1, df2].

Worse, this answer even states the opposite: "If you just add to the list, it won't delete the original dataframe, when you delete the list", and concludes that one should prefer del df1 to del [df1].

How can there be a difference between these calls to del, when the language definition suggests they are the same?

PLNech
  • 3,087
  • 1
  • 23
  • 52

1 Answers1

2

There is literally no difference whatsoever. They do the same thing. They even compile to completely identical bytecode:

In [1]: import dis

In [2]: dis.dis('del x')
  1           0 DELETE_NAME              0 (x)
              2 LOAD_CONST               0 (None)
              4 RETURN_VALUE

In [3]: dis.dis('del [x]')
  1           0 DELETE_NAME              0 (x)
              2 LOAD_CONST               0 (None)
              4 RETURN_VALUE

In [4]: dis.dis('del [[x]]')
  1           0 DELETE_NAME              0 (x)
              2 LOAD_CONST               0 (None)
              4 RETURN_VALUE

The results you saw were most likely due to execution order effects.

user2357112
  • 260,549
  • 28
  • 431
  • 505