2

I couldn't find a convenient way to distinguish in-place methods from assignable methods in python. I mean for example a method like my_list.sort() don't need assignment and it does changes itself (it is in-place right?), but some other methods need assignment to a variable. am I wrong ?

Hooman Talakian
  • 51
  • 1
  • 7
  • 1
    I'm not sure what you're talking about. By "in place", do you mean methods that mutate the object they are called upon? – khelwood Aug 15 '21 at 18:12
  • 2
    There's a foolproof method: read the documentation. If the documentation is unclear, fix it – Mad Physicist Aug 15 '21 at 18:13
  • 1
    Usually, object methods like `x.sort()` will mutate it, and free functions like `sorted(x)` will return a new object. – interjay Aug 15 '21 at 18:14
  • 3
    @interjay. Usually is an awfully strong term in this case – Mad Physicist Aug 15 '21 at 18:34
  • 4
    By *convention* and the general rule of thumb is if the data type is mutable, like `list`s, the methods are in-place, but if they are not like strings, then they return new instances. The documentation is always the authoritative source, of course. – martineau Aug 15 '21 at 19:42

1 Answers1

1

The reason you can't easily find such a distinction is that it really doesn't exist except by tenuous convention. "In-place" just means that a function modifies the data of a mutable argument, rather than returning an all new object. If "Not in-place" is taken to mean that a function returns a new object encapsulating updated data, leaving the input alone, then there are simply too many other possible conventions to be worth cataloging.

The standard library does its best to follow the convention of not returning values from single-argument in-place functions. So for example you have list.sort, list.append, random.shuffle and heapq.heapify all operate in-place, returning None. At the same time, you have functions and methods that create new objects, and must therefore return them, like sorted, list.__add__ and tuple.__iadd__. But you also have in-place methods that must return a value like list.__iadd__ (compare to list.extend which does not return a value).

__iadd__ and similar in-place operators emphasize a very important point, which is that in-place operation is not an option for immutable objects. Python has a workaround for this:

x = (1, 2)
y = (3, 4)
x += y

For all objects, the third line is equivalent to

x = type(x).__iadd__(x, y)

Ignoring the fact that the method is called as a function, notice that the name x is reassigned, so even if x += y has to create a new object (e.g., because tuple is immutable), you can still see it through the name x. Mutable objects will generally just return x in this case, so the method call will appear not to return a value, even when it really does.

As an interesting aside, the reassignment sometimes causes an unexpected error:

>>> z = ([],)
>>> z[0].extend([1, 2])  # OK
>>> z[0] += [3, 4]  # Error! But list is mutable!

Many third party libraries, such as numpy, support the convention of in-place functions without a return value, up to a point. Most numpy functions create new objects, like np.cumsum, np.add, and np.sort. However, there are in also functions and methods that operate in-place and return None, like np.ndarray.sort and np.random.shuffle.

numpy can work with large memory buffers, which means that in-place operation is often desirable. Instead of having a separate in-place version of the function, some functions and methods (most notably universal functions) have an out parameter that can be set to the input, like np.cumsum, np.ndarray.cumsum, and np.add. In these cases, the function will operate in-place, but still return a reference to the out parameter, much in the same way that python's in-place operators do.

An added complication is that not all functions and methods perform a single action on a single object. You can write a class like this to illustrate:

class Test:
    def __init__(self, value):
        self.value = value
    def op(self, other):
         other.value += self.value
         return self

This class modifies another object, but returns a reference to the unmodified self. While contrived, the example serves to illustrate that the in-place/not-in-place paradigm is not all-encompassing.

TL;DR

In the end, the general concept of in-place is often useful, but can't replace the need for reading documentation and understanding what each function does on an individual basis. This will also save you from many common gotchas with mutable objects supporting in-place operations vs immutable ones just emulating them.

Mad Physicist
  • 107,652
  • 25
  • 181
  • 264