How to speed up Python code using numpy?

Question

Orginal and Mutated are images. I need to get the difference of each r,g,b separately. I got this code to work, but it is to slow. Any help on making this fast would be nice! :)

Orginal = np.asarray(Orginal).copy()
Mutated = np.asarray(Mutated).copy()    

Fittnes = 0

for x in range(0, 299):

    for y in range(0, 299):

        DeltaRed   = (Orginal[x][y][0] - Mutated[x][y][0])
        DeltaGreen = (Orginal[x][y][1] - Mutated[x][y][1])
        DeltaBlue  = (Orginal[x][y][2] - Mutated[x][y][2])

        Fittnes += (DeltaRed * DeltaRed + DeltaGreen * DeltaGreen + DeltaBlue * DeltaBlue)

return Fittnes

You want to redo python part? Because there is a way to speed-up numpy itself, using intel MKL https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl — Severin Pappadeux, Nov 15 '15 at 02:09
@SeverinPappadeux MKL would only be relevant for linear algebra ops, e.g. matrix products. Simple vectorized subtraction and summation are handled by numpy's internals rather than external libraries. — ali_m, Nov 15 '15 at 11:59

score 3 · Answer 1 · answered Nov 15 '15 at 02:20

3

It should be a whole lot faster if you didn't go the extra mile of zipping and then summing up each dimension instead of using numpy's sum function:

DeltaRed   = np.sum(OR) - np.sum(MR)
DeltaGreen = np.sum(OG) - np.sum(MG)
DeltaBlue = np.sum(OB) - np.sum(MB)

answered Nov 15 '15 at 02:20

Carsten

17,991
4
48
53

Any way to improve it more? – Funktiona Nov 15 '15 at 02:25
On my computer, this gives you at least a 100x speed improvement. For anything beyond I'd have to know abot your particular use case. If you're summing up the whole image, all additional improvements will be small. – Carsten Nov 15 '15 at 02:33

Divakar · Answer 2 · 2015-11-15T18:03:32.290

1

Here's one approach to do all those in one summation with ndarray.sum -

DeltaRed, DeltaGreen, DeltaBlue = Orginal.sum((0,1)) - Mutated.sum((0,1))

Here's another with np.einsum and hopefully faster one, when working with uint8 images -

org_diff = np.einsum('ijk->k',Orginal.astype('uint64'))
mut_diff = np.einsum('ijk->k',Mutated.astype('uint64'))
DeltaRed, DeltaGreen, DeltaBlue = org_diff - mut_diff

edited Nov 15 '15 at 18:03

answered Nov 15 '15 at 08:43

Divakar

218,885
19
262
358

This gets me the total RGB changes, I need them to calculate the RGB difference for each pixel, and get the total RGB changes for all pixels combined. – Funktiona Nov 15 '15 at 17:36
@Funktiona ` RGB difference for each pixel` would be `Orginal - Mutated`, right? And `total RGB changes for all pixels combined` would be as listed in the solution? – Divakar Nov 15 '15 at 17:40
Look my code bellow. They does not do the same as your solutions – Funktiona Nov 15 '15 at 17:42
@Funktiona What's the datatype of `Orginal` and `Mutated`? – Divakar Nov 15 '15 at 17:46
they are numpy.asarray – Funktiona Nov 15 '15 at 17:48
@Funktiona That's not a datatype. You can find out the datatype with `Orginal.dtype` and `Mutated.dtype`. – Divakar Nov 15 '15 at 17:49
@Funktiona I think when working with huge arrays, the solution with `np.einsum` would have inconsistencies, so might be a better idea to use the `.sum((0,1))` based approach instead. – Divakar Nov 15 '15 at 18:11
What do you mean by "inconsistencies" – Funktiona Nov 15 '15 at 18:18
@Funktiona Output values being different. – Divakar Nov 15 '15 at 18:21
Its still not what I want. This gets me the sum of all red combined in original - all red combined in mutated. What I want is to calculate the RGB difference for each pixel. Lets say Original[10, 10] has the RGB [100, 100, 100]. And Mutated[10, 10] has the RGB [180, 180, 180]. The output for this pixel should me. ((100 - 180) * (100 - 180) + (100 - 180) * (100 - 180) + (100 - 180) * (100 - 180)). So I get a value on how similar the mutated pixel is compared to the Orginal. I need to multiply it by itself so I don’t get negative numbers – Funktiona Nov 15 '15 at 18:39
@Funktiona `((Orginal - Mutated)**2).sum((0,1))`? – Divakar Nov 15 '15 at 18:43
If two pixels in Mutated swaps possession, don't I still get the same value? This is function decides the similarities between them. I need to compare them pixel for pixel, not just compare the total RGB difference. – Funktiona Nov 15 '15 at 19:00
@Funktiona If you have a for-loop code that works please edit the question. Then, we can think of optimizing it. – Divakar Nov 15 '15 at 19:02
@Funktiona The edited code is same as your [`previous question`](http://stackoverflow.com/q/33709902/3293881) right? So, didn't the solution to that previous question work for you? – Divakar Nov 15 '15 at 19:16
@Funktiona You did confirm on the previous question's solution that it worked with sum : `((Orginal - Mutated)**2).sum()`. Now , you are saying it's not working? – Divakar Nov 15 '15 at 19:27
After some more testing I got that it diden't. Sorry – Funktiona Nov 15 '15 at 19:28
@Funktiona I think your code is wrong because as you said you are using `uint8` datatype and you are subtracting and multiplying, in which you are encountering overflows. It's getting too long with these conversations, so good luck, sorry can't help you any further as we are moving in circles here. – Divakar Nov 15 '15 at 19:34

score 0 · Answer 3 · answered Nov 15 '15 at 17:40

This was the code from beginning that worked.

    Fittnes = 0

    for x in range(0, 299):

        for y in range(0, 299):

            DeltaRed   = (Orginal[x][y][0] - Mutated[x][y][0])
            DeltaGreen = (Orginal[x][y][1] - Mutated[x][y][1])
            DeltaBlue  = (Orginal[x][y][2] - Mutated[x][y][2])

            Fittnes += (DeltaRed * DeltaRed + DeltaGreen * DeltaGreen + DeltaBlue * DeltaBlue)

   return Fittnes

How to speed up Python code using numpy?

3 Answers3