Working with raw file on Python. ValueError: cannot reshape array of size 5038848 into shape

Question

I wrote code in Python programming language. This code should first open the raw file and output it as an image. Next, you need to split the pixels into RGB values and calculate the 3x3 matrix for color correction. But before that, you need to do a black level correction. The picture shows the approximate operation of the algorithm. Raw file is 14 bit and has RG pattern. Link to raw file: text enter image description here The code:

import numpy as np
from PIL import Image

def open_raw_file(raw_file_path, width, height):
    with open(raw_file_path, 'rb') as file:
        raw_data = file.read()

    # Assuming raw data is in RGB format, each pixel consists of 3 bytes (R, G, B)
    # Adjust the data format based on your specific raw file format if necessary
    image_data = np.frombuffer(raw_data, dtype=np.uint8)
    image_data = image_data.reshape((height, width, 3))

    return image_data

def apply_black_level_correction(image_data, black_level):
    corrected_image_data = np.maximum(image_data - black_level, 0)
    return corrected_image_data

def split_into_channels(image_data):
    red_channel = image_data[:, :, 0]
    green_channel = image_data[:, :, 1]
    blue_channel = image_data[:, :, 2]
    return red_channel, green_channel, blue_channel

def calculate_color_correction_matrix(image_data):
    # Assuming a 3x3 color correction matrix is required
    # Calculate the matrix based on your specific algorithm or requirements
    color_correction_matrix = np.eye(3)  # Identity matrix for demonstration purposes
    return color_correction_matrix

def save_image(image_data, output_path):
    image = Image.fromarray(image_data)
    image.save(output_path)

# Main code
raw_file_path = 'macbeth_2592x1944_RG.raw'
output_image_path = 'output_image13.jpg'
width = 640  # Adjust as per your raw file dimensions
height = 480  # Adjust as per your raw file dimensions
black_level = 20  # Adjust the black level value as per your requirement

# Open raw file and convert it into an image
image_data = open_raw_file(raw_file_path, width, height)

# Apply black level correction
corrected_image_data = apply_black_level_correction(image_data, black_level)

# Split into RGB channels
red_channel, green_channel, blue_channel = split_into_channels(corrected_image_data)

# Calculate color correction matrix
color_correction_matrix = calculate_color_correction_matrix(corrected_image_data)

# Perform color correction
red_channel_corrected = np.dot(red_channel, color_correction_matrix[0])
green_channel_corrected = np.dot(green_channel, color_correction_matrix[1])
blue_channel_corrected = np.dot(blue_channel, color_correction_matrix[2])

# Merge color channels back into image
output_image_data = np.stack((red_channel_corrected, green_channel_corrected, blue_channel_corrected), axis=2)

# Save the corrected image
save_image(output_image_data, output_image_path)

I got this error:

Traceback (most recent call last):
  File "C:\Users\DC\PycharmProjects\pythonProject4\lab13.py", line 43, in <module>
    image_data = open_raw_file(raw_file_path, width, height)
  File "C:\Users\DC\PycharmProjects\pythonProject4\lab13.py", line 11, in open_raw_file
    image_data = image_data.reshape((height, width, 3))
ValueError: cannot reshape array of size 5038848 into shape (480,640,3)

So i try to run this code with:

1. As I said raw file is 14 bit, but I run it with 8 bit (didnt help).
2. I tried with size: w:640, h:480 and w:2592, h:1944 (original raw size) (Didnt help).
3. I tried to change reshape function changing image_data = image_data.reshape((height, width, 3)) to image_data = image_data.reshape(height, width). None of that helped me.

Answer 1

The image you have shared is 10,077,696 bytes, and assuming the filename is correct and the image is 2592x1944, that means it has two bytes per pixel because

2592 x 1944 x 2 = 10,077,696

So, if I unpack it as 16-bit greyscale with ImageMagick like this:

convert -depth 16 -size 2592x1944 gray:macbeth_2592x1944_RG.raw -verbose info:

I obtain:

Image:
  Filename: gray:macbeth_2592x1944_RG.raw
  Format: GRAY (Raw gray samples)
  Class: DirectClass
  Geometry: 2592x1944+0+0
  Units: Undefined
  Colorspace: Gray
  Type: Grayscale
  Endianness: Undefined
  Depth: 16-bit
  Channel depth:
    gray: 15-bit                    <--- HERE
  Channel statistics:
    Pixels: 5038848
    Gray:
      min: 808  (0.0123293)         <--- HERE
      max: 16256 (0.248051)         <--- HERE
      mean: 3913.4 (0.0597147)
...
...

I can now see it contains 15-bits of legitimate data per pixel and the greyscale range of the samples is 808..16256.

That would make it dark, as nothing exceeds 25% of the full 16-bit range of 0..65535. So I would apply auto-levels to correct the contrast and make it into JPEG like this:

convert -depth 16 -size 2592x1944 gray:macbeth_2592x1944_RG.raw -auto-level a.jpg

---

It therefore looks like 2592x1944 is the correct geometry, and there are 16-bits per pixel. The name implies the 16-bits might be 8-bits of red and 8-bits of green or some Bayer-like mash-up of the two. The black level of 800 in the description looks like it is correct because the range of the grey levels is 808..16256, so that would mean the correct range is 8..15,456.

---

Coming back to your Python, that would mean you need to use:

width = 2592 
height = 1944

and the following to read 2 bytes per pixel and reshape correctly:

image_data = np.frombuffer(raw_data, dtype=np.uint16).reshape((height, width))

Answer 2

We may figure out the image format from the readme.txt and from the file name.

Content of readme.txt:

Bayer pattern = RG
black level = 800
bpp = 16
bit depth = 14

We may conclude that the image is in Bayer CFA format (probably RGGB Bayer alignment).
The black level is 800.
There are 16 bits per pixel (we may assume each pixel is uint16 element).
The white level is 2**14-1 = 16383 (applying bit depth = 14).

The file name is macbeth_2592x1944_RG.raw.
We may conclude that the resolution is 2592x1944 (width=2592, height=1944).

---

In Bayer mosaic format, each pixel may be either red, green or blue (it's kind of grayscale image format, but the filter on each pixel determines the color).
For converting Bayer to RGB, we may use OpenCV cv2.cvtColor method.
The process of converting from Bayer to RGB is named Demosaicing.
The input to the Demosaicing algorithm is single channel image, and the output is 3 channels RGB image.

---

Reading the raw image - make sure to read 2592x1944 pixels with uint16 type:

width = 2592
height = 1944

def open_raw_file(raw_file_path, width, height):
    with open(raw_file_path, 'rb') as file:
        raw_data = file.read()

    image_data = np.frombuffer(raw_data, dtype=np.uint16)  # According to the readme.txt bpp = 16 (16 bits per pixel).
    image_data = image_data.reshape(height, width)  # The image is in raw Bayer format - before Demosaicing there is only one channel.

---

After the black level correction apply Demosaicing:

rgb_image = cv2.cvtColor(corrected_image_data, cv2.COLOR_BayerBG2RGB)  # COLOR_BayerBG2RGB gives the best results.

---

There is an issue with the way color correction is applied, but since the matrix is an eye matrix, let's skip the details.

---

Before saving the image as JPEG format, we have to convert the image to type uint8 (pixel range [0, 255]).
We may apply linear transformation followed by rounding, clipping and casting:

while_level = 2**14 - 1  # bit depth = 14
output_image_data_8bpp = output_image_data.astype(np.float32) * (255 / (while_level - black_level))  # Scale to range [0, 255]
output_image_data_8bpp = output_image_data_8bpp.round().clip(0, 255).astype(np.uint8)  # Convert to uint8 with rounding and clipping. (JPEG format must be uint8).

---

Updated code sample:

import numpy as np
import cv2
from PIL import Image

def open_raw_file(raw_file_path, width, height):
    with open(raw_file_path, 'rb') as file:
        raw_data = file.read()

    # Assuming raw data is in RGB format, each pixel consists of 3 bytes (R, G, B)
    # Adjust the data format based on your specific raw file format if necessary
    #image_data = np.frombuffer(raw_data, dtype=np.uint8)
    image_data = np.frombuffer(raw_data, dtype=np.uint16)  # According to the readme.txt bpp = 16 (16 bits per pixel).
    #image_data = image_data.reshape((height, width, 3))
    image_data = image_data.reshape(height, width)  # The image is in raw Bayer format - before Demosaicing there is only one channel.

    return image_data

def apply_black_level_correction(image_data, black_level):
    corrected_image_data = np.maximum(image_data - black_level, 0)
    return corrected_image_data

def split_into_channels(image_data):
    red_channel = image_data[:, :, 0]
    green_channel = image_data[:, :, 1]
    blue_channel = image_data[:, :, 2]
    return red_channel, green_channel, blue_channel

def calculate_color_correction_matrix(image_data):
    # Assuming a 3x3 color correction matrix is required
    # Calculate the matrix based on your specific algorithm or requirements
    color_correction_matrix = np.eye(3)  # Identity matrix for demonstration purposes
    return color_correction_matrix

def save_image(image_data, output_path):
    image = Image.fromarray(image_data)
    image.save(output_path)

# Main code
raw_file_path = 'macbeth_2592x1944_RG.raw'
output_image_path = 'output_image13.jpg'

# The file name is macbeth_2592x1944_RG.raw so the width is 2592 and the height is 1944
width = 2592  # 640  # Adjust as per your raw file dimensions
height = 1944  # 480  # Adjust as per your raw file dimensions
black_level = 800 #20  # Note: According readme.txt black level equals 800 # Adjust the black level value as per your requirement

# Open raw file and convert it into an image
image_data = open_raw_file(raw_file_path, width, height)

# Apply black level correction (it's fine to apply black level correction before Demosaicing).
corrected_image_data = apply_black_level_correction(image_data, black_level)

# The file name is macbeth_2592x1944_RG so the Bayer patten is RGGB.
# Apply Demosaicing - convert from Bayer CFA to RGB
rgb_image = cv2.cvtColor(corrected_image_data, cv2.COLOR_BayerBG2RGB)  # COLOR_BayerBG2RGB gives the best results.

# Split into RGB channels
#red_channel, green_channel, blue_channel = split_into_channels(rgb_image) # (corrected_image_data)

# Calculate color correction matrix
color_correction_matrix = calculate_color_correction_matrix(corrected_image_data)

# Perform color correction
# Color correction scale [r, g, b] triplet by column vector (or row vector by [r, g, b] triplet)
red_channel_corrected = np.dot(rgb_image, color_correction_matrix[0])  #np.dot(red_channel, color_correction_matrix[0])
green_channel_corrected = np.dot(rgb_image, color_correction_matrix[1])  #np.dot(green_channel, color_correction_matrix[1])
blue_channel_corrected = np.dot(rgb_image, color_correction_matrix[2])  #np.dot(blue_channel, color_correction_matrix[2])

# Merge color channels back into image
output_image_data = np.stack((red_channel_corrected, green_channel_corrected, blue_channel_corrected), axis=2)

# According to readme.txt bit depth = 14
# We may assume that the white level is 2**14-1 = 16383.
# Convert from 16 bpp to 8 bits in range [0, 255] using linear transformation:  
while_level = 2**14 - 1  # bit depth = 14
output_image_data_8bpp = output_image_data.astype(np.float32) * (255 / (while_level - black_level))  # Scale to range [0, 255]
output_image_data_8bpp = output_image_data_8bpp.round().clip(0, 255).astype(np.uint8)  # Convert to uint8 with rounding and clipping. (JPEG format must be uint8).

# Save the corrected image
save_image(output_image_data_8bpp, output_image_path)

---

Output image (downscaled):

Note:
The output is greenish due to the lack of color correction (this is normal).