0

I have a 2D matrix. I want to compare each value with the mean of the values in a box around it(excluding the value itself). If the value is bigger than the mean plus 3sigma it will be replaced by the mean of the values in the box. Then I repeat the process over the matrix five times. So, I have these pice of code that works just fine in my little example matrix, but when I go to the real one (750X2000) it takes forever to run. Is there any way to speed up this loop? Im relatively new to python, so maybe there are some basic, obvious things that Im not seeing.

    import numpy as np
    mat= [[ 0 , 2 , 4 , 60000 , 8],[10 ,12, 14000, 16, 18],[20 ,22, 625, 26, 28],[30, 32,34, 36, 38],[10000, 42, 44, 46, 48]] 
    mat=np.array(mat)
    print(mat)
    print(mat.shape[0])
    print('###############')
    b=0
    while b<5:
          for i in range(mat.shape[0]):
               for j in range(mat.shape[1]):
                   chunk = mat[max(0,i-1):i+2, max(0,j-1):j+2]
                   c = np.setdiff1d(chunk,mat[i,j])
                   media_c=np.mean(c)
                   #print(media_c)
                   sdv=np.std(c)
                   if mat[i,j]>media_c+3*sdv:
                       mat[i,j]=media_c
        
          b+=1
    
     print(mat)
1201ProgramAlarm
  • 32,384
  • 7
  • 42
  • 56
Alv
  • 1
  • 1

1 Answers1

1

I would do it a little bit differently:

  1. you should use sum pooling for this with same padding (so the output is same size), best to do that on GPU with tensorflow, sum pooling is not defined but you can easily do with average pooling and then multiply by number of fields of filter, e.g. 9 (3x3), you can of course do it in numpy as well, I have no experience with that.

  2. after that, you can subtract original image from sum-pooled, so you get difference, very fast operation

  3. there you can compare with your sigma, I would use argwhere in numpy or where in tensorflow, way faster than iterating over each element

This is just a concept, I have no time to create code snippets now, will maybe add later if needed.

Ruli
  • 2,592
  • 12
  • 30
  • 40