Is there a convenient way to apply a lookup table to a large array in numpy?

Question

I’ve got an image read into numpy with quite a few pixels in my resulting array.

I calculated a lookup table with 256 values. Now I want to do the following:

for i in image.rows:
    for j in image.cols:
        mapped_image[i,j] = lut[image[i,j]]

Yep, that’s basically what a lut does.
Only problem is: I want to do it efficient and calling that loop in python will have me waiting for some seconds for it to finish.

I know of numpy.vectorize(), it’s simply a convenience function that calls the same python code.

Answer 1

You can just use image to index into lut if lut is 1D.
Here's a starter on indexing in NumPy:
http://www.scipy.org/Tentative_NumPy_Tutorial#head-864862d3f2bb4c32f04260fac61eb4ef34788c4c

In [54]: lut = np.arange(10) * 10

In [55]: img = np.random.randint(0,9,size=(3,3))

In [56]: lut
Out[56]: array([ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90])

In [57]: img
Out[57]: 
array([[2, 2, 4],
       [1, 3, 0],
       [4, 3, 1]])

In [58]: lut[img]
Out[58]: 
array([[20, 20, 40],
       [10, 30,  0],
       [40, 30, 10]])

Mind also the indexing starts at 0

Answer 2

TheodrosZelleke's answer in correct, but I just wanted to add a little undocumented wisdom to it. Numpy provides a function, np.take, which according to the documentation "does the same thing as fancy indexing."

Well, almost, but not quite the same:

>>> import numpy as np
>>> lut = np.arange(256)
>>> image = np.random.randint(256, size=(5000, 5000))
>>> np.all(lut[image] == np.take(lut, image))
True
>>> import timeit
>>> timeit.timeit('lut[image]',
...               'from __main__ import lut, image', number=10)
4.369504285407089
>>> timeit.timeit('np.take(lut, image)',
...               'from __main__ import np, lut, image', number=10)
1.3678052776554637

np.take is about 3x faster! In my experience, when using 3D luts to convert images from RGB to other color spaces, adding logic to convert the 3D look-up to a 1D flattened look-up allows a x10 speed up.

Answer 3

If you are limited to using numpy, TheodrosZelleke's answer is the way to go. But if you allow other modules, cv2 is a useful module for interacting with image data, and it accepts numpy arrays as input. A big limitation is that the image array must have dtype='uint8', but as long as that is OK, the function cv2.LUT does exactly what we want, and it provides a significant speedup:

>>> import numpy as np
>>> import cv2
>>> lut = np.arange(256, dtype='uint8')
>>> image = np.random.randint(256, size=(5000, 5000), dtype='uint8')
>>> np.all(lut[image] == cv2.LUT(image, lut))
True
>>> import timeit
>>> timeit.timeit('lut[image]', 'from __main__ import lut, image', number=10)
0.5747578000000431
>>> timeit.timeit('cv2.LUT(image, lut)', 
...               'from __main__ import cv2, lut, image', number=10)
0.07559149999997317

Your lookup table can be some other datatype, but you loose a lot of the speed improvement (although numpy indexing takes a performance hit as well). For example, with dtype='float64':

>>> lut = np.arange(256, dtype='float64')
>>> timeit.timeit('lut[image]', 'from __main__ import lut, image', number=10)
1.068468699999812
>>> timeit.timeit('cv2.LUT(image, lut)', 
...               'from __main__ import cv2, lut, image', number=10)
0.41085720000000947