Optimizing performance when reading a satellite image file in python

Question

I have a multiband satellite image stored in the band interleaved pixel (BIP) format along with a separate header file. The header file provides the details such as the number of rows and columns in the image, and the number of bands (can be more than the standard 3).

The image itself is stored like this (assume a 5 band image):

[B1][B2][B3][B4][B5][B1][B2][B3][B4][B5] ... and so on (basically 5 bytes - one for each band - for each pixel starting from the top left corner of the image).

I need to separate out each of these bands as PIL images in Python 3.2 (on Windows 7 64 bit), and currently I think I'm approaching the problem incorrectly. My current code is as follows:

def OpenBIPImage(file, width, height, numberOfBands):
    """
    Opens a raw image file in the BIP format and returns a list
    comprising each band as a separate PIL image.
    """
    bandArrays = []
    with open(file, 'rb') as imageFile:
        data = imageFile.read()
    currentPosition = 0
    for i in range(height * width):
        for j in range(numberOfBands):
            if i == 0:
                bandArrays.append(bytearray(data[currentPosition : currentPosition + 1]))
            else:
                bandArrays[j].extend(data[currentPosition : currentPosition + 1])
            currentPosition += 1
    bands = [Image.frombytes('L', (width, height), bytes(bandArray)) for bandArray in bandArrays]
    return bands

This code takes way too long to open a BIP file, surely there must be a better way to do this. I do have the numpy and scipy libraries as well, but I'm not sure how I can use them, or if they'll even help in any way.

Since the number of bands in the image are also variable, I'm finding it hard to figure out a way to read the file quickly and separate the image into its component bands.

And just for the record, I have tried messing with the list methods in the loops (using slices, not using slices, using only append, using only extend etc), it doesn't particularly make a difference as the major time is lost because of the number of iterations involved - (width * height * numberOfBands).

Any suggestions or advice would be really helpful. Thanks.

Answer 1

If you can find a fast function to load the binary data in a big python list (or numpy array), you can de-interleave the data using the slicing notation:

band0 = biglist[::nbands]
band1 = biglist[1::nbands]
....

Does that help?

Answer 2

Standard PIL

To load an image from a file, use the open function in the Image module.

>>> import Image
>>> im = Image.open("lena.ppm")

If successful, this function returns an Image object. You can now use instance attributes to examine the file contents.

>>> print im.format, im.size, im.mode
PPM (512, 512) RGB

The format attribute identifies the source of an image. If the image was not read from a file, it is set to None. The size attribute is a 2-tuple containing width and height (in pixels). The mode attribute defines the number and names of the bands in the image, and also the pixel type and depth. Common modes are "L" (luminance) for greyscale images, "RGB" for true colour images, and "CMYK" for pre-press images.

The Python Imaging Library also allows you to work with the individual bands of an multi-band image, such as an RGB image. The split method creates a set of new images, each containing one band from the original multi-band image. The merge function takes a mode and a tuple of images, and combines them into a new image. The following sample swaps the three bands of an RGB image:

Splitting and merging bands

r, g, b = im.split()
im = Image.merge("RGB", (b, g, r))

So I think you should simply derive the mode and then split accordingly.

PIL with Spectral Python (SPy python module)

However, as you pointed out in your comments below, you are not dealing with a normal RGB image with 3 bands. So to deal with that, SpectralPython (a pure python module which requires PIL) might just be what you are looking for.

Specifically - http://spectralpython.sourceforge.net/class_func_ref.html#spectral.io.bipfile.BipFile

spectral.io.bipfile.BipFile deals with Image files with Band Interleaved Pixel (BIP) format.

Hope this helps.

Answer 3

I suspect that the repetition of extend is not good better allocate all first

def OpenBIPImage(file, width, height, numberOfBands):
    """
    Opens a raw image file in the BIP format and returns a list
    comprising each band as a separate PIL image.
    """
    bandArrays = []
    with open(file, 'rb') as imageFile:
        data = imageFile.read()
    currentPosition = 0
    for j in range(numberOfBands):
        bandArrays[j]= bytearray(b"\0"*(height * width)):


    for i in xrange(height * width):
        for j in xrange(numberOfBands):
                bandArrays[j][i]=data[currentPosition])
            currentPosition += 1
    bands = [Image.frombytes('L', (width, height), bytes(bandArray)) for bandArray in bandArrays]
    return bands

my measurements doesn't show nsuch a slow down

def x():
    height,width,numberOfBands=1401,801,6
    before = time.time()
    for i in range(height * width):
        for j in range(numberOfBands):
            pass
    print (time.time()-before)

>>> x()
0.937999963760376

EDITED