0

I am developing an Image viewer.
My main goal is to make something able to easily load .EXR files along with all other main formats.

I based it around :

  • PySide2 (Qt) for the UI.
  • OpenImageIO for .exr files (and some others) loading. I built a custom python module based around OpenImageIO (made with C++ and PyBind11)

On the c++ side, the output is pretty straight forward, my custom module return a simple struct
( nicely 'converted' into python by pybind11) :

struct img_data
{
    bool is_valid;
    int width;
    int height;
    int nchannels;
    std::vector<unsigned char> pixels;
};

It's less straightforward when in python:

# get image_data from custom module
image_data = custom_exr_loader.load_image(image_path)

# convert pixels to a numpy array 
np_array = np.uint8(image_data.pixels)
np_array.shape = (image_data.height, image_data.width, image_data.nchannels)

# convert to a PIL Image
pil_image = Image.fromarray(np_array).convert("RGBA")

# convert to a QImage
image_qt : ImageQt = ImageQt(pil_image)

# and finally convert to a QPixmap for final consumption ...
pixmap = QPixmap.fromImage(image_qt, Qt.ImageConversionFlag.ColorOnly)

Performance is pretty bad. Especially when caching a whole directory of big EXRs

I bet there is a better way to do this ...
Of course the idea is to offload some of the work from python to C++ land. But I'm not sure how.
I tried using QImage.fromData() method, but I never got a working result. It requires a QByteArray as a param.

How should I format pixels data in c++ to be able to use QImage.fromData() from UI code ?

wohlstad
  • 12,661
  • 10
  • 26
  • 39
gui2one
  • 160
  • 1
  • 1
  • 14
  • 2
    Why are you passing through PIL? Besides, if you can work in C++, can't you create the QImage directly there? – musicamante Aug 07 '22 at 09:26
  • 1
    To extend what written above: from your question it's not really clear, but, assuming that `custom_exr_loader` is what gets the struct above, then you could easily implement it on the C++ side. You already have everything you need (image size, channels, pixel data), so you can easily create a QImage from there using [one of the existing constructors](https://doc.qt.io/qt-5/qimage.html#QImage-5), which would be something like `QImage(pixels, width, height, width * nchannels, QImage::Format_RGBA8888)`. Then return the created QImage to Python. – musicamante Aug 07 '22 at 20:03
  • It looks like exactly what I need. Frankly, I didn't realize I could use Qt in C++ without using the full blown thing ... But how do I load it once in python ? In a bytearray or something ? QImage( data_array ) ? – gui2one Aug 08 '22 at 07:20
  • I don't know how PyBind works, but, as far as I can understand, you should just be able to return the QImage from the C++ function, and PyQt/PySide should be able to wrap it as a python object. – musicamante Aug 09 '22 at 13:07

1 Answers1

1

The problem here are the copies of the data. You are actually using 5 representations:

  • mage_data from load_image
  • numpy array
  • pil image with conversion
  • QImage
  • QPixmap again with conversion

All your conversion as far as I can see make a complete copy of the image. Sometimes you might even convert the data. This is detrimental for performance.

What you need is are converters that don't make a copy but just memory map the data in some array. Ideally you would need to do this for your binding as well. I am not sure how PyBind11 works, but you need to somehow give access to the raw C++ memory used there. Also you might be able to use QImage from the C++ side as such a wrapper. For numpy you need to use np.frombuffer. To go to Qt you can do something like

im_np = np.array(img)    
qimage = QImage(im_np.data, im_np.shape[1], im_np.shape[0],                                                                                                                                                 
                 QImage.Format_BGR888)

Also you need to find a representation (column, pixel format) that is supported by all those entities. It's not an easy task. You would need to research a copy free mapping for each of your conversion. Examples are qimage_from_numpy, library_qimage2ndarray, pil_from_numpy. And there are many more.

Chris
  • 2,461
  • 1
  • 23
  • 29
  • You're right, loading pixels directly into a numpy array is way more straightforward, the conversion I did to a PIL Image was totally unnecessary. Thank you, performance is already better. – gui2one Aug 08 '22 at 07:23