Python: Convert RAW image to PNG

Question

I am struggling with processing a raw image with Python to convert it to png format. Processing these type of data is new to me and I don't quite understand the logic yet.

The image data (as per the camera doc) is 12 bit, little-Endian order with a Bayer pattern. The image is 2048x1944 px. Image available here.

I have followed @Rotem answer in here but the image I get is basically just noise. Code below.

Any idea on how to approach this?

import cv2
import numpy as np

width = 2048
height = 1944

with open('.raw_image.raw', "rb") as rawimg:
    # Read the packed 12bits as bytes - each 3 bytes applies 2 pixels
    data = np.fromfile(rawimg, np.uint8, width * height * 3//2)

    data = data.astype(np.uint16)  # Cast the data to uint16 type.
    result = np.zeros(data.size*2//3, np.uint16)  # Initialize matrix for storing the pixels.

    # 12 bits packing: ######## ######## ########
    #                  | 8bits| | 4 | 4  |  8   |
    #                  |  lsb | |msb|lsb |  msb |
    #                  <-----------><----------->
    #                     12 bits       12 bits
    result[0::2] = ((data[1::3] & 15) << 8) | data[0::3]
    result[1::2] = (data[1::3] >> 4) | (data[2::3] << 4)
    bayer_im = np.reshape(result, (height, width))

    # Apply Demosacing (COLOR_BAYER_BG2BGR gives the best result out of the 4 combinations).
    bgr = cv2.cvtColor(bayer_im, cv2.COLOR_BAYER_BG2BGR)  # The result is BGR format with 16 bits per pixel and 12 bits range [0, 2^12-1].
    # bgr = cv2.cvtColor(bayer_im, cv2.COLOR_BayerGB2BGR)  # The result is BGR format with 16 bits per pixel and 12 bits range [0, 2^12-1].
    # bgr = cv2.cvtColor(bayer_im, cv2.COLOR_BayerRG2BGR)  # The result is BGR format with 16 bits per pixel and 12 bits range [0, 2^12-1].
    # bgr = cv2.cvtColor(bayer_im, cv2.COLOR_BayerGR2BGR )  # The result is BGR format with 16 bits per pixel and 12 bits range [0, 2^12-1].

    # Show image for testing (multiply by 16 because imshow requires full uint16 range [0, 2^16-1]).
    cv2.imshow('bgr', cv2.resize(bgr*16, [width//10, height//10]))
    cv2.waitKey()
    cv2.destroyAllWindows()

    # Convert to uint8 before saving as JPEG (not part of the conversion).
    colimg = np.round(bgr.astype(float) * (255/4095))
    cv2.imwrite("./test.png", colimg)

A link to the camera and its documentation might be useful. – Mark Setchell Mar 25 '22 at 09:07 — Mark Setchell, Mar 25 '22 at 09:07

score 2 · Accepted Answer · answered Mar 25 '22 at 10:27

The raw image you have posted is 12 bit per pixel but without packing.
The 12 bits data are stored in the higher 12 bits of every 16 bits.

We can see by the size of the file, that there are 2 bytes per pixel:
7962624 = 2048*1944*2

We may represent the 12 bits data in uint16 element as follows:

 ------------------------------------------------------------------------------- 
|    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |    |
| b11| b10| b9 | b8 | b7 | b6 | b5 | b4 | b3 | b2 | b1 | b0 |  0 |  0 |  0 |  0 |
 -------------------------------------------------------------------------------

This format is much more simple to work with compared to the 12 bits packed format...
We don't need to unpack the data, we may look at it as 16 bits per pixel.

Code sample:

import cv2
import numpy as np

width = 2048
height = 1944

with open("raw_image.ims_rgb", "rb") as rawimg:
    # Read the raw image as uint16 (two bytes per pixel).
    bayer_im = np.fromfile(rawimg, np.uint16, width * height).reshape(height, width)

    # The 12 bits of each pixel are stored in the upper 12 bits of every uint16 element.
    # The lower 4 bits of the uint16 element are zeros.
    # <--- 16 bits -->
    # ************0000
    # <-12 bits -><4->    
    #    data     zeros

    # Apply Demosacing.
    # It look like COLOR_BAYER_BG2BGR gives the best result, but it hard to tell from the given input.
    bgr = cv2.cvtColor(bayer_im, cv2.COLOR_BAYER_BG2BGR)  # The result is BGR format with 16 bits per pixel range [0, 2^16-1].

    # Apply manual "white balance".
    # The result image is greenish - this is normal for most cameras.
    # We may fix it by scaling up the red and the blue color channels.
    # It look like approximate scaling is about 1.5 for the red and about 1.25 for the blue.
    bgr[:, :, 0] = (bgr[:, :, 0].astype(np.float32)*1.25).clip(0, 65535).astype(np.uint16)
    bgr[:, :, 2] = (bgr[:, :, 2].astype(np.float32)*1.5).clip(0, 65535).astype(np.uint16)


    # Show image for testing (multiply by 16 because imshow requires full uint16 range [0, 2^16-1]).
    cv2.imshow('bgr', cv2.resize(bgr, [width//10, height//10]))
    cv2.waitKey()
    cv2.destroyAllWindows()

    # Save the output as tiff with 16 bits per color component.
    cv2.imwrite("rgb16.tif", bgr)

We can see, it's a picture of the moon:

The moon is not the best choice, because we can't verify the correctness of the colors...

Note:
I "manually" scaled up the red and the blue color channels to make the moon gray (instead of green).
We may refer the scaling as manual White Balance.

Thanks @Rotem, you're awesome! Your solutions works perfect! — davipeix, Mar 25 '22 at 11:38
By any chance, is there any alternative to do the demosaicing not using opencv, say to keep libraries limited to numpy? I have tried to take each channel from the `bayer_im` matrix and do a bilinear inference with numpy to infer the 3 channels at full resolution. However. the results were not as expected. As a note, the Bayer patters is GRBG. — davipeix, Apr 04 '22 at 16:40
You can implement the paper [HIGH-QUALITY LINEAR INTERPOLATION FOR DEMOSAICING OF BAYER-PATTERNED COLOR IMAGES](https://stanford.edu/class/ee367/reading/Demosaicing_ICASSP04.pdf). It is not so difficult to implement with NumPy. — Rotem, Apr 04 '22 at 16:48

Python: Convert RAW image to PNG

1 Answers1