Updating referenced variables in functions in python

Question

I'm trying to change the order of the matrix X through class P, which is already passed to a function in the base class B. The change in order does not reflect in the base class B. Is there a way to achieve this? Please see the following MWE:

#p.py
import numpy as np
import b

class P(b.B):

    def __init__(self, opts):
        self.opts = opts

    def func1(self, X):

        print('func1:',X)

    def func2(self):

        randomize = np.arange(len(self.myX))
        np.random.shuffle(randomize)
        self.myX = self.myX[randomize]

        print('func2',self.myX)

    def func3(self, X):
        """
        X is a 2d matrix 
        """
        self.myX = X
        #call the function from the base class 
        b.B.func3(self, self.myX)  

#b.py
class B:
    """
    base class.

    """
    def __init__(self, opts):
        self.opts = opts


    def func3(self, X):

        for n in range(2):
            for i in range(X.shape[0]):
                self.func1(X[i])
            self.func2()

from the console:

p1 = p.P({})
X=np.array([[1,2,3], [2,3,4], [3,4,5], [4,5,6], [5,6,7], [6,7,8]])
p1.func3(X)

Current Output:

func1: [1 2 3]
func1: [2 3 4]
func1: [3 4 5]
func1: [4 5 6]
func1: [5 6 7]
func1: [6 7 8]
func2 [[6 7 8]
 [3 4 5]
 [2 3 4]
 [5 6 7]
 [4 5 6]
 [1 2 3]]
func1: [1 2 3]
func1: [2 3 4]
func1: [3 4 5]
func1: [4 5 6]
func1: [5 6 7]
func1: [6 7 8]

Expected Output:

func1: [1 2 3]
func1: [2 3 4]
func1: [3 4 5]
func1: [4 5 6]
func1: [5 6 7]
func1: [6 7 8]
func2 [[6 7 8]
 [3 4 5]
 [2 3 4]
 [5 6 7]
 [4 5 6]
 [1 2 3]]
func1: [6 7 8]
func1: [3 4 5]
func1: [2 3 4]
func1: [5 6 7]
func1: [4 5 6]
func1: [1 2 3]

So basically, when the control returns from p.func2 to func3 in B, X should be the same as self.myX. I believe this should happen because self.myX is by default passed by reference to b.func3.

Can you clarify which behaviour you get and which you expect? Running your code unmodified correctly calls the methods of ``P`` for me. — MisterMiyagi, Nov 15 '18 at 15:43
Did you mean you expected `B.func3()` to call upon `B.func1()` and `B.func2()` instead of `P.func1()` and `P.func2()`? It's not clear what you're asking. — r.ook, Nov 15 '18 at 15:59
You are actually overwriting all the methods from the base class `B` so I'm not sure the point of you using `b.B` as a base class. You don't even need that base class since you are directly referencing `b.B.func3` in your `p.P.func3` anyways. If you mean to keep the methods of `b.B` you would need to name the functions differently. "Changes not reflecting in base class `B`" is also confusing because you really only created a `P` instance with the base `B`. There's no instance of `B` for you to observe any reflection - everything happens under `P()`. — r.ook, Nov 15 '18 at 16:08
@Idlehands this is a minimum working version of a big code. I just want to get this problem fixed so did not feel necessary to include other details of the functions. — user1340852, Nov 15 '18 at 16:31
Possible duplicate of [How do I pass a variable by reference?](https://stackoverflow.com/questions/986006/how-do-i-pass-a-variable-by-reference) — kabanus, Nov 15 '18 at 17:29

r.ook · Accepted Answer · 2018-11-15T17:26:31.280

The problem is here:

#class p.P
    def func2(self):

        randomize = np.arange(len(self.myX))
        np.random.shuffle(randomize)
        self.myX = self.myX[randomize]  # <-- this line

        print('func2',self.myX)

You effectively reassigned self.myX to a new randomized np array, changing the object reference away from X to this new randomized array.

If you added a print(id(self.myX)) between this line, you'll notice the initial reference points back to X, but once you reassigned it, the id is not longer the same.

You should do this instead to maintain the reference:

np.random.shuffle(self.myX)

If you actually wanted to shuffle the inner array as well:

for arr in self.myX:
    np.random.shuffle(arr)
np.random.shuffle(self.myX)

This will also maintain the reference.

Edit: If you wish to keep a reference of the randomized order while still maintaining the object, it's a bit trickier but doable:

# use make randomize an instance attribute so you can refer to it even after the function ended
self.shuffle_order = np.arange(len(self.myX))
np.random.shuffle(self.shuffle_order)

# loop through each inner array and reassign based on a copy of the shuffled matrix
for i, arr in enumerate(self.myX[self.shuffle_order]):
    self.myX[i] = arr

This changes the inner array but maintain the overall object reference between self.myX and X.

Afterwards you can retrieve the order via p1.shuffle_order.

Thank you for your response. This really works! However, I want to save the order in which they are randomized to shuffle other corresponding arrays. Is that possible? — user1340852, Nov 15 '18 at 16:59
I've updated the answer with a take on referencing the shuffled order afterwards. — r.ook, Nov 15 '18 at 17:27

Updating referenced variables in functions in python

1 Answers1