How to match the color models of BufferedImage and Mat?

Question

I'm trying to convert BufferedImage to Mat for a large set of images with different file types downloaded from the internet. Because I am scraping the images from websites, I have no control over the file formats. I can easily load them to BufferedImage without knowing the format, but to convert them to Mat, I need to know the image type. Unfortunately, there doesn't seem to be a nice correspondence between CvType and BufferedImage types.

CvType represents image types with the format CV_<bit-depth>{U|S|F}C<number_of_channels> where U is unsigned char, S is signed char, and F is float.

BufferedImage types have more variety in the representation including a symmetrical channels (TYPE_4BYTE_ABGR), varying numbers of bits (TYPE_BYTE_BINARY), and whatever an indexed byte image is (TYPE_BYTE_INDEXED).

Based on the documentation, I tried to complete my own correspondence.

BufferedImage imgBuffer = ImageIO.read(new File("example.gif"));

//Save file as reference
File outputfile = new File("temp/image.png");
ImageIO.write(imgBuffer, "png", outputfile);

//My correspondance
int curCVtype = -1;
switch (imgBuffer.getType()) {
case BufferedImage.TYPE_3BYTE_BGR:
    curCVtype = CvType.CV_8UC3;
    break;
case BufferedImage.TYPE_BYTE_GRAY:
    curCVtype = CvType.CV_8UC1;
    break;
case BufferedImage.TYPE_INT_BGR:
case BufferedImage.TYPE_INT_RGB:
    curCVtype = CvType.CV_8SC3;
    break;
case BufferedImage.TYPE_INT_ARGB:
case BufferedImage.TYPE_INT_ARGB_PRE:
    curCVtype = CvType.CV_8SC4;
    break;
default:
//  The types not handled by my correspondence
//                  BufferedImage.TYPE_BYTE_BINARY;
//                  BufferedImage.TYPE_USHORT_GRAY;
//                  BufferedImage.TYPE_4BYTE_ABGR;
//                  BufferedImage.TYPE_4BYTE_ABGR_PRE;
//                  BufferedImage.TYPE_BYTE_INDEXED;
//                  BufferedImage.TYPE_CUSTOM;

    System.out.println("Unsupported format:" + imgBuffer.getType());
    //Here I choose a default type
    curCVtype = CvType.CV_8SC3;
}

//Convert to Mat
byte[] pixels = ((DataBufferByte) imgBuffer.getRaster().getDataBuffer()).getData();
Mat img = new Mat(imgBuffer.getHeight(), imgBuffer.getWidth(), curCVtype);
img.put(0, 0, pixels);

//Write the output to compare
Imgcodecs.imwrite("temp/image_mat.png", img);

Questions

Am I going about this correctly? Or is there a better approach?
What are the correct correspondences for the types that are in my default case?

Examples

Input

Converted to PNG by BufferedImage

Output PNG after conversion to Mat. BufferedImage type wasTYPE_BYTE_INDEXED converted to CvType.CV_8SC3

Output PNG after conversion to Mat. BufferedImage type wasTYPE_BYTE_INDEXED converted to CvType.CV_8UC3

Resources:

My starting code came from Converting BufferedImage to Mat in opencv .

What I know about CvTypes came from What's the difference between cvtype values in OPENCV? .

TYPE_INT should map to CV_32S, not CV_8S. Then TYPE_BYTE_BINARY to CV_8UC1, TYPE_USHORT_GRAY to CV_16UC1, TYPE_4BYTE_ABGR to CV_8UC4. And use CV_8UC3 as default — Miki, Oct 28 '15 at 23:55
@Miki I initially thought the same about the int types, but was later confused by the documentation which says "Represents an image with 8-bit RGB color components packed into integer pixels." Just to clarify, the colors only require 8-bits, but they are stored as 32-bits? — Cecilia, Oct 29 '15 at 00:00
_packed into integer pixels_ , so you need CV_32S. I don't know BufferedImage, but I think that you can convert all buffered images to RGB. Then you can just convert to opencv as if type was TYPE_3BYTE_BGR — Miki, Oct 29 '15 at 00:05
@Miki Thanks, I think your comments would make a good answer. — Cecilia, Oct 29 '15 at 00:08
Note that `TYPE_CUSTOM` isn't a single type, it is used to represent everything that is not one of the other types. You could probably create a direct conversion for all the known types, then a fallback using `getRGB` to `CV_8UC4` or `CV_32U` (I don't think any of the Java `TYPE_INT_xxx` should be treated as signed) which will be slower, but still fully compatible. — Harald K, Oct 29 '15 at 07:55
Have you tried just loading them from the file system directly to a Mat with Imgcodecs.imread() (opencv 3.0) or its equivalent in 2.4.x? Or do you need to get them into a BufferedImage first? — medloh, Oct 29 '15 at 16:32
@medloh BufferedImage has more supported file formats then Imgcodecs.imread(). Specifically, Imgcodecs.imread() cannot handle .gif — Cecilia, Oct 29 '15 at 16:35

Cecilia · Accepted Answer · 2015-10-29T17:59:05.037

Thanks to Miki and haraldK for the helpful comments.

My solution for unknown image types retrieves the pixels in RGB format, and put them into a Mat of CvType.CV_8UC4 . Finally, reorder the channels using Core.mixChannels to the OpenCV prefered order: BGR(A).

This example only reorders the channels for the unknown image types, but all non BGR image types would need to be reordered.

BufferedImage imgBuffer = ImageIO.read(new File("example.gif"));

//Save image as reference
File outputfile = new File("temp/image.png");
ImageIO.write(imgBuffer, "png", outputfile);

//My correspondance

int curCVtype = CvType.CV_8UC4; //Default type
boolean supportedType = true;

switch (imgBuffer.getType()) {
case BufferedImage.TYPE_3BYTE_BGR:
    curCVtype = CvType.CV_8UC3;
    break;
case BufferedImage.TYPE_BYTE_GRAY:
case BufferedImage.TYPE_BYTE_BINARY:
    curCVtype = CvType.CV_8UC1;
    break;
case BufferedImage.TYPE_INT_BGR:
case BufferedImage.TYPE_INT_RGB:
    curCVtype = CvType.CV_32SC3;
    break;
case BufferedImage.TYPE_INT_ARGB:
case BufferedImage.TYPE_INT_ARGB_PRE:
    curCVtype = CvType.CV_32SC4;
    break;
case BufferedImage.TYPE_USHORT_GRAY:
    curCVtype = CvType.CV_16UC1;
    break;
case BufferedImage.TYPE_4BYTE_ABGR:
case BufferedImage.TYPE_4BYTE_ABGR_PRE:
    curCVtype = CvType.CV_8UC4;
    break;
default:
    // BufferedImage.TYPE_BYTE_INDEXED;
    // BufferedImage.TYPE_CUSTOM;
    System.out.println("Unsupported format:" + imgBuffer.getType());
    supportedType = false;
}

//Convert to Mat
Mat img = new Mat(imgBuffer.getHeight(), imgBuffer.getWidth(), curCVtype);
if (supportedType) {
    // Insert pixel buffer directly
    byte[] pixels = ((DataBufferByte) imgBuffer.getRaster().getDataBuffer()).getData();
    img.put(0, 0, pixels);
} else {
    // Convert to RGB first
    int height = imgBuffer.getHeight();
    int width = imgBuffer.getWidth();
    int[] pixels = imgBuffer.getRGB(0, 0, width - 1, height - 1, null, 0, width);

    // Convert ints to bytes
    ByteBuffer byteBuffer = ByteBuffer.allocate(pixels.length * 4);
    IntBuffer intBuffer = byteBuffer.asIntBuffer();
    intBuffer.put(pixels);

    byte[] pixelBytes = byteBuffer.array();

    img.put(0, 0, pixelBytes);

    // Reorder the channels for Opencv BGRA format from
    // BufferedImage ARGB format
    Mat imgMix = img.clone();
    ArrayList<Mat> imgSrc = new ArrayList<Mat>();
    imgSrc.add(imgMix);

    ArrayList<Mat> imgDest = new ArrayList<Mat>();
    imgDest.add(img);

    int[] fromTo = { 0, 3, 1, 2, 2, 1, 3, 0 }; //Each pair is a channel swap
    Core.mixChannels(imgSrc, imgDest, new MatOfInt(fromTo));
}

//Save output image
Imgcodecs.imwrite("temp/image_mat.png", img);

The new output image

I'm using Javacv there is no Matofint(int...a) function in it. — Aqeel Haider, Sep 21 '16 at 12:24
@AqeelHaider The standard Java bindings to include it (http://docs.opencv.org/java/2.4.9/org/opencv/core/MatOfInt.html) — Cecilia, Sep 21 '16 at 17:12
I use IplImage to load image. use cvResize() method to resize it. Convert the Image to buffered Image to make the image corners round. and use TYPE_INT_ARGB Then convert it back to Mat to do perspective Transform. convert it back to buffered image to merge with another image. and then save it. Problem come when i convert image from buffer to Mat its colour change. — Aqeel Haider, Sep 21 '16 at 18:44
if i use TYPE_3BYTE_BGR in buffered image and then convert it Mat then colors didn't change, but it remove the background transparency of image to back. — Aqeel Haider, Sep 21 '16 at 18:52

How to match the color models of BufferedImage and Mat?

Questions

Examples

Resources:

1 Answers1