Numpy - how to build a new image from 4 input images, picking the brightest pixels (max(R+G+B))

Question

There are four image arrays like this:

image1 = np.array([
    [
      # [R  G  B]
      #  |  |  |
        [1, 2, 3], [11, 22, 55], [12, 45, 56]
    ],
    [
        [1, 2, 3], [56, 55, 13], [12, 45, 56]
    ],
    [
        [11, 22, 55], [56, 55, 13], [12, 45, 56]
    ],
])

image2 = np.array([
    [
        [91, 72, 33], [111, 222, 155], [212, 245, 156]
    ],
    [
        [100, 200, 113], [56, 255, 213], [112, 145, 156]
    ],
    [
        [113, 223, 255], [156, 55, 113], [212, 245, 156]
    ],
])

image3 = np.array([
    [
        [9, 2, 3], [111, 222, 255], [22, 25, 16]
    ],
    [
        [10, 20, 13], [156, 25, 23], [12, 45, 16]
    ],
    [
        [13, 23, 155], [56, 255, 13], [222, 235, 216]
    ],
])

image4 = np.array([
    [
        [29, 22, 23], [111, 222, 255], [223, 125, 216]
    ],
    [
        [210, 220, 13], [156, 252, 232], [122, 145, 216]
    ],
    [
        [123, 232, 155], [56, 255, 213], [222, 235, 216]
    ],
])

For each rows and columns, I want to compute the max RGB value of sum of Red & Green & Blue channel of the image arrays and use the max to build a new image array.

So after tried the following code, It got duplicate indices, because in each rows and columns the max RGB of three image arrays may be duplicate. But I just want to get the first match index.

array = np.array([image1, image2, image3, image4])
array_sum = array.sum(axis=3)
# array_sum_max_index = array_sum.argmax(axis=0)
indices = np.where(array_sum == array_sum.max(axis=0))
print(indices)

Output

(array([1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 3]), array([0, 0, 2, 0, 2, 0, 1, 1, 1, 2, 2]), array([0, 2, 0, 1, 2, 1, 0, 1, 2, 1, 2]))

There is a similar answer in this link, but it cannot resolve my question. So how to get the index when it first match.

Edit:

The actual number of input images is more than 32. (Solutions involving numpy.choose() may not work because of this)

How does `array_sum_max_index` or `array_sum.max(axis=0)` not give you what you want? It looks correct to me, but perhaps I am misunderstanding your goal. `array_sum_max_index` gives `array([[1, 2, 1], [3, 3, 3], [1, 3, 2]])` telling you which image is brightest at each pixel, and `array_sum.max(axis=0)` gives you the brightest pixel taken from all the images at each location. — John Zwinck, Jan 26 '21 at 09:18
I want to generate a new array use the max `RGB`, but how to access `array` by `array_sum_max_index` is hard to me. — luneice, Jan 26 '21 at 09:56
The line `array_sum_max_index = array_sum.argmax(axis=0)`, which you have commented out from some reason, gives you exactly the `(3,3)` "new image array" which you have asked for in your question. — fountainhead, Jan 26 '21 at 10:05
Yes, `array_sum_max_index` is right answer, but I do not know how to use it to access `array`. So I tried `where` to get indices, which return the indices. — luneice, Jan 26 '21 at 10:59

score 2 · Answer 1 · answered Jan 26 '21 at 12:57

(Proposing an alternative solution that does not use numpy.choose(), since OP has indicated that the constraint of 32 choice arrays in the case of numpy.choose() is an issue)

array = np.array([image1, image2, image3, image4])
array_sum = array.sum(axis=3)
array_sum_max_index = array_sum.argmax(axis=0)

m = 3 # Number of pixel rows
n = 3 # Number of pixel columns

array[array_sum_max_index, np.arange(m)[:,None,], np.arange(n)[None,:]]

Output:

array([[[ 91,  72,  33],
        [111, 222, 255],
        [212, 245, 156]],

       [[210, 220,  13],
        [156, 252, 232],
        [122, 145, 216]],

       [[113, 223, 255],
        [ 56, 255, 213],
        [222, 235, 216]]])

Daniel F · Answer 2 · 2021-01-26T13:40:51.763

2

In one line (based on @fountainhead 's answer):

np.take_along_axis(array, array.sum(3).argmax(0)[None, ..., None], 0)

Out[]: 
array([[[[ 91,  72,  33],
         [111, 222, 255],
         [212, 245, 156]],

        [[210, 220,  13],
         [156, 252, 232],
         [122, 145, 216]],

        [[113, 223, 255],
         [ 56, 255, 213],
         [222, 235, 216]]]])

edited Jan 26 '21 at 13:40

answered Jan 26 '21 at 13:26

Daniel F

13,620
2
29
55

Nice. And to think that all these (learning) days, I've been looking around for a real-world usecase for `numpy.take_along_axis()` ! – fountainhead Jan 26 '21 at 13:43
It's made for this use-case, but it can be pretty clunky with the requirement that the array and indices have the same number of dimensions (thus the `[None, ..., None]`) – Daniel F Jan 26 '21 at 13:45

score 1 · Answer 3 · answered Jan 26 '21 at 10:05

1

You can do it using np.choose:

np.choose(array_sum_max_index, array_sum)

Which gives you the brightest (highest R+G+B) pixel at each location:

array([[196, 588, 613],
       [443, 640, 483],
       [591, 524, 673]])

The result is the same as array_sum.max(axis=0).

answered Jan 26 '21 at 10:05

John Zwinck

239,568
38
324
436

I think `np.choose(array_sum_max_index, array)` is what's really required. – fountainhead Jan 26 '21 at 10:25
No, its not quite. In the `array_sum_max_index` return, which each rows and columns number represents the max `RGB` index of the `array`. So, I can generate a new array by `array_sum_max_index` return, but its hard to me. – luneice Jan 26 '21 at 10:56

fountainhead · Accepted Answer · 2021-01-26T11:54:40.083

1

I think this is what you're looking for:

np.choose(array_sum_max_index[...,None], tuple(array))

Output:

array([[[ 91,  72,  33],
        [111, 222, 255],
        [212, 245, 156]],

       [[210, 220,  13],
        [156, 252, 232],
        [122, 145, 216]],

       [[113, 223, 255],
        [ 56, 255, 213],
        [222, 235, 216]]])

Explanation:

array_sum_max_index has the shape (m,n), where each image is an array of shape (m,n,3)
array is an array of shape (4,m,n,3), and each "element" of tuple(array) is a sub-array of shape (m,n,3)
We're trying to use np.choose(), passing array_sum_max_index as a "selector" array, and tuple(array) as the sequence of sub-arrays from which the selector will select from. But as you can see in the documentation for numpy.choose(), the "selector" array, and the sub-arrays being chosen from, must all be broadcastable together. Since the "selector" array array_sum_max_index is of shape (m,n), to make it broadcastable with sub-arrays of shape (m,n,3), we append an extra dimension of unit length to array_sum_max_index, using the expression array_sum_max_index[...,None]
This would still work if we used just array instead of tuple(array), but the documentation for numpy.choose() discourages that.

edited Jan 26 '21 at 11:54

answered Jan 26 '21 at 11:14

fountainhead

3,584
1
8
17

Yes, but the `numpy.choose()` allows at least 0 and at most 32 array objects to pass. So the shape (x, ...) of `array`, the `x` must be between 0 and 32. – luneice Jan 26 '21 at 12:00
@luneice -- Do you intend to pass more than 32 images to `np.choose()` to choose from ? – fountainhead Jan 26 '21 at 12:07
Yes, when passing more than 32 images, it got `ValueError: Need at least 0 and at most 32 array objects`. – luneice Jan 26 '21 at 12:08
1

In that case, you'll have wrap all this code into a function, and call the function multiple times. So, if you have 50 images, the first call to your function would return a new image that uses the brightest pixels from the first 32 images. The next call to your function must pass this returned image and the remaining 18 images, to get the final image that uses the brightest pixels from all 50 images. – fountainhead Jan 26 '21 at 12:12
1

@luneice - I've posted a different answer that does not use `numpy.choose()`. Please take a look. – fountainhead Jan 26 '21 at 13:01

Numpy - how to build a new image from 4 input images, picking the brightest pixels (max(R+G+B))

4 Answers4