0

I have a 3D numpy array

dark = np.array(dark_ref.load())
dark.shape
(100, 384, 249)

I have a list of indices for the second and third dimension

l, r, b = np.where(dark > 9000)
len(r)
1799
len(b)
1799

and I have written this function to replace the values at indices r and b with an average of the adjacent values

def flatten_bad_band(A, x, y): # x is 3d array, y is bad coords
    testarr = A.copy()
    for a in range(A.shape[0]):
        for i in range(A.shape[1]):
            for j in range((A.shape[2])):
                testarr[a, x[i], y[j]] = (testarr[a, x[i+1], y[j]] + testarr[a, x[i-1], y[j]]) /2 
    return testarr
test = flatten_bad_band(dark, r, b)

This seems to work, but it is slow...., and I will need to use this on much larger arrays. Is there a better way to do this? I am new to python, numpy and coding!

russj
  • 25
  • 1
  • 6
  • I don't think your code works! you index your vectors `r, b` in the range of the shape of your matrix, i.e. `0, 384` for r and `0, 249` for b, but your vectors are `1799` long. So I am not really sure what your code does or is supposed to do. But, in case this already helps: Instead of iterating through all your matrix dimensions, iterate over all elements of your indices: `for x,y in zip(l, b):` you save some loops that way at least. – Nyquist Sep 27 '22 at 11:49
  • You are right, the code as written does not do what I thought it does! Thanks for the suggestion, will see if I can get it working – russj Sep 27 '22 at 12:02

2 Answers2

0

I have a 3D numpy array

dark = np.array(dark_ref.load())
dark.shape
(100, 384, 249)

I have a list of indices for the second and third dimension

l, r, b = np.where(dark > 9000)
len(r)
1799
len(b)
1799

and I have written this function to replace the values at indices r and b with an average of the adjacent values

def flatten_bad_band(A, x, y): # x is 3d array, y is bad coords
    testarr = A.copy()
    for a in range(A.shape[0]):
        for i, j in zip(x, y):
                testarr[a, x[i], y[j]] = (testarr[a, x[i+1], y[j]] + testarr[a, x[i-1], y[j]]) /2 
    return testarr
dark_clean = flatten_bad_band(dark, r, b)

commenter @Nyquist, suggested an improvement to skip one loop and pointed out an error that meant the code wasnt working as intended. fixed with his suggestion and seems to work faster too

russj
  • 25
  • 1
  • 6
0

You can also eliminate the first for loop and let numpy do all the work along the axis. Don't know if this is faster, but since numpy has a C backend, it is more than likely that it is! It is also good practice.

Besides that, you should look into python's for loops! Your code still doesn't work as you attempt to index your list with elements of your list.

Here is an example, eliminating all unnecessary loops and also deals with cases where you attempt to average out of bounds elements. Hope this helps.

import numpy as np

Atest = np.random.randint(0,10, (5,5,5)).astype(float)

l = [1,2]
r = [2,3]

def flatten_bad_band(A, l, r):
    resarr = A.copy()
    testarr = np.pad(resarr, ((0,0),(1,1),(0,0)), 'constant', constant_values=0)
    
    for idx in zip(l, r):
        resarr[:,idx[0], idx[1]] = np.average(np.stack((testarr[:,idx[0], idx[1]], testarr[:,idx[0]+2, idx[1]])), axis=0)

    return resarr

print(flatten_bad_band(Atest, l, r))
Nyquist
  • 328
  • 1
  • 6
  • You are correct, I am trying to index my list with elements of my list, when what I want to do is index my array with element of my list. I dont understand your code, leave it with me. Thanks for new response! – russj Sep 27 '22 at 13:22
  • Perfect, pulled it apart and played with it to see how it works, and it does exactly what I want. Thanks! – russj Sep 27 '22 at 16:48