create new numpy array base on exist numpy array after many caculate step

Question

i have a 2D array numpy s like this:


s = np.array([[ 5.,  4., np.nan,  1., np.nan],
       [np.nan,  4.,  4.,  2.,  2.],
       [ 3., np.nan, np.nan,  5.,  5.],
       [np.nan,  3.,  4.,  4., np.nan]])

#now i want to create a new np array s1 base on s like this

s1= np.empty((len(s),len(s)))
for i  in range(len(s)):
    a = np.abs(s - s[i])
    a = np.nanmean(a, axis=1)
    w = 1 / (a + 0.001)
    s1[i] = w
s1
array([[1000.        ,    1.99600798,    0.33322226,    0.49975012],
       [   1.99600798, 1000.        ,    0.33322226,    0.999001  ],
       [   0.33322226,    0.33322226, 1000.        ,    0.999001  ],
       [   0.49975012,    0.999001  ,    0.999001  , 1000.        ]])

#without use for loop i write like this 

def f(x,y):
    a = np.abs(s[y]-s[x])
    a = np.nanmean(a)
    if np.isnan(a):
        return 0
    w = 1/(a+0.001) #not let 1/0
    return w

s1 = np.fromfunction(np.vectorize(f),(len(s),len(s)),dtype='int')
s1
array([[1000.        ,    1.99600798,    0.33322226,    0.49975012],
       [   1.99600798, 1000.        ,    0.33322226,    0.999001  ],
       [   0.33322226,    0.33322226, 1000.        ,    0.999001  ],
       [   0.49975012,    0.999001  ,    0.999001  , 1000.        ]])

First i want to ask is my np.fromfunction right? Second, are there another ways to rewrite this code with numpy without use for loop?

score 1 · Answer 1 · answered Dec 03 '21 at 04:25

1

You may be looking for a pure numpy solution:

repeat_s_by_array = np.repeat(s[np.newaxis,:,:], repeats=len(s), axis=0) 
repeat_s_by_rows = np.repeat(s, repeats=len(s), axis=0).reshape(len(s),len(s),-1)
abs_vals = np.abs(repeat_s_by_array - repeat_s_by_rows)
mean = np.nanmean(abs_vals, axis=-1)
s1 = 1 / (mean + 0.001)

Or you can do a one-liner:

s1 = 1 / (np.array([np.nanmean(np.abs(s - s[i]), axis=1) for i in range(len(s))]) + 0.001)

answered Dec 03 '21 at 04:25

if i divide s rows to many part, call a is the number of row. should i change repeats from len(s) to a? your code seem work with small size of s – robocon20x Dec 03 '21 at 07:15
1

If your job is the same as in your original question, it must be `repeats=len(a)`. But because numpy solution deals with a much larger array, if `s` has many rows, then your original loop construction or the one-liner in my answer may begin to perform better. – Dec 03 '21 at 08:02

keepAlive · Accepted Answer · 2021-12-03T10:18:35.627

1

What about

>>> 1/(
        1e-3 + np.nanmean(
            np.abs(s - s[:, None,:]),
            axis=2
        )
    )
array([[1.00000000e+03, 1.99600798e+00, 3.33222259e-01, 4.99750125e-01],
       [1.99600798e+00, 1.00000000e+03, 3.33222259e-01, 9.99000999e-01],
       [3.33222259e-01, 3.33222259e-01, 1.00000000e+03, 9.99000999e-01],
       [4.99750125e-01, 9.99000999e-01, 9.99000999e-01, 1.00000000e+03]])

Or in a fully numpy-idiomatic fashion

>>> np.reciprocal(1e-3 + np.nanmean(np.abs(s - s[:, np.newaxis, :]), axis=2))
array([[1.00000000e+03, 1.99600798e+00, 3.33222259e-01, 4.99750125e-01],
       [1.99600798e+00, 1.00000000e+03, 3.33222259e-01, 9.99000999e-01],
       [3.33222259e-01, 3.33222259e-01, 1.00000000e+03, 9.99000999e-01],
       [4.99750125e-01, 9.99000999e-01, 9.99000999e-01, 1.00000000e+03]])

_{where, as of December 2021, np.newaxis is None.}

edited Dec 03 '21 at 10:18

answered Dec 03 '21 at 09:50

keepAlive

6,369
5
24
39

thank Sir, you save me 1 more time. Where can i read document about create a new axis in numpy, i read np.newaxis in nympy doc but it so basic,... and i really dont know how it work – robocon20x Dec 03 '21 at 10:03
1

You are welcome @robocon20x Indeed, you are right: if you master newaxis-like stuffs, your NumPy-life is gonna be much easier. A first (good) link to grasp the thing is *[How does numpy.newaxis work and when to use it?](https://stackoverflow.com/a/41267079/4194079)*. – keepAlive Dec 03 '21 at 10:07

create new numpy array base on exist numpy array after many caculate step

2 Answers2