Python - writing an in-place option in my function [how to prevent overwriting my input vector]

Question

I am writing a function that pulls the top x values from a sparse vector (fewer values if there are less than x). I would like to include an "in-place" option like many functions have, where it removes the top values if the option is True, and keeps them if the option is False.

My issue is that my current function is overwriting the input vector, rather than keeping it as is. I am not sure why this is occurring. I expected the way to solve my design problem would be to include an if statement which would copy the input using copy.copy(), but this raises a value error (ValueError: row index exceeds matrix dimensions) which does not make sense to me.

Code:

from scipy.sparse import csr_matrix
import copy

max_loc=20
data=[1,3,3,2,5]
rows=[0]*len(data)
indices=[4,2,8,12,7]

sparse_test=csr_matrix((data, (rows,indices)), shape=(1,max_loc))

print(sparse_test)

def top_x_in_sparse(in_vect,top_x,inplace=False):
    if inplace==True:
        rvect=in_vect
    else:
        rvect=copy.copy(in_vect)
    newmax=top_x
    count=0
    out_list=[]
    while newmax>0:
        newmax=1
        if count<top_x:
            out_list+=[csr_matrix.max(rvect)]
            remove=csr_matrix.argmax(rvect)
            rvect[0,remove]=0
            rvect.eliminate_zeros()
            newmax=csr_matrix.max(rvect)
            count+=1
        else:
            newmax=0
    return out_list

a=top_x_in_sparse(sparse_test,3)

print(a)

print(sparse_test)

My question has two parts:

1. how do I prevent this function from overwriting the vector?
2. how do I add the inplace option?

Answer 1

You really just don't want to loop period. Reallocating those arrays every loop iteration with .eliminate_zeros() is the slowest thing but not the only reason not do to it.

import numpy as np
from scipy.sparse import csr_matrix

max_loc=20
data=[1,3,3,2,5]
rows=[0]*len(data)
indices=[4,2,8,12,7]

sparse_test=csr_matrix((data, (rows,indices)), shape=(1,max_loc))

Something like this would be better:

def top_x_in_sparse(in_vect,top_x,inplace=False):
    
    n = len(in_vect.data)
    
    if top_x >= n:
        if inplace:
            out_data = in_vect.data.tolist()
            in_vect.data = np.array([], dtype=in_vect.data.dtype)
            in_vect.indices = np.array([], dtype=in_vect.indices.dtype)
            in_vect.indptr = np.array([0, 0], dtype=in_vect.indptr.dtype)
            return out_data
        else:
            return in_vect.data.tolist()
        
    else:
        k = n - top_x
        partition_idx = np.argpartition(in_vect.data, k)

        if inplace:
            out_data = in_vect.data[partition_idx[k:n]].tolist()
            in_vect.data = in_vect.data[partition_idx[0:k]]
            in_vect.indices = in_vect.indices[partition_idx[0:k]]
            in_vect.indptr = np.array([0, len(in_vect.data)], dtype=in_vect.indptr.dtype)            
            return out_data
        else:
            return in_vect.data[partition_idx[k:n]].tolist()
    

If you need the returned values sorted you can do that as well of course.

a=top_x_in_sparse(sparse_test,3,inplace=False)

>>> print(a)
[3, 5, 3]

>>> print(sparse_test)
  (0, 2)    3
  (0, 4)    1
  (0, 7)    5
  (0, 8)    3
  (0, 12)   2

b=top_x_in_sparse(sparse_test,3,inplace=True)

>>> print(b)
[3, 5, 3]

>>> print(sparse_test)
  (0, 4)    1
  (0, 12)   2

Also as per your question from the comments: a shallow copy of the sparse array object will not copy the numpy arrays that hold the data. The sparse object only has a reference to those objects. A deep copy would get them, but using the built-in method for copy already takes care of knowing which things that are referenced need to be copied and which dont.