Converting `BufferedImage` to `Mat` in OpenCV

Question

How can I convert a BufferedImage to a Mat in OpenCV?

I'm using the JAVA wrapper for OpenCV(not JavaCV). As I am new to OpenCV I have some problems understanding how Mat works.

I want to do something like this. (Based on Ted W. reply):

BufferedImage image = ImageIO.read(b.getClass().getResource("Lena.png"));

int rows = image.getWidth();
int cols = image.getHeight();
int type = CvType.CV_16UC1;
Mat newMat = new Mat(rows, cols, type);

for (int r = 0; r < rows; r++) {
    for (int c = 0; c < cols; c++) {
        newMat.put(r, c, image.getRGB(r, c));
    }
}

Highgui.imwrite("Lena_copy.png", newMat);

This doesn't work. Lena_copy.png is just a black picture with the correct dimensions.

is it your custom java wrapper ? for `javacv` it is just `IplImage.createFrom(bufferedImage)`. please post some code where u stucked ? — Nikson Kanti Paul, Feb 19 '13 at 13:38
Opencv has it's own Java binding as of version 2.4.4. http://opencv.org/opencv-java-api.html — Jompa234, Feb 19 '13 at 21:51

andriy · Accepted Answer · 2013-03-05T10:12:15.773

30

I also was trying to do the same thing, because of need to combining image processed with two libraries. And what I’ve tried to do is to put byte[] in to Mat instead of RGB value. And it worked! So what I did was:

1.Converted BufferedImage to byte array with:

byte[] pixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();

2. Then you can simply put it to Mat if you set type to CV_8UC3

image_final.put(0, 0, pixels);

Edit: Also you can try to do the inverse as on this answer

edited Mar 05 '13 at 10:12

answered Feb 26 '13 at 17:39

andriy

4,074
9
44
71

7

Well, there is something that viktorich didn't say in his answer. The BufferedImage object must be declared like this: `img = new BufferedImage(320, 240, BufferedImage.TYPE_3BYTE_BGR);`
`img.setRGB(0, 0, 320, 240, intArray, 0, 320);` In other way, the above code will not work. – Kevin Infante May 02 '14 at 23:10
The code will work in other cases, too. Both formats have to match, for example `BufferedImage.TYPE_BYTE_GRAY` and `CV_8UC(1)` works as well. – Mene Mar 16 '15 at 17:18
1

What is the version of this library this works for? It doesn't work for me (Mat class doesn't have this method put with such signature), I'm using bytedeco javacv 1.3.2. – Sergey Karpushin Jul 11 '17 at 14:45

Sumsuddin Shojib · Answer 2 · 2018-09-09T10:39:06.017

Don't want to deal with big pixel array? Simply use this

BufferedImage to Mat

public static Mat BufferedImage2Mat(BufferedImage image) throws IOException {
    ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
    ImageIO.write(image, "jpg", byteArrayOutputStream);
    byteArrayOutputStream.flush();
    return Imgcodecs.imdecode(new MatOfByte(byteArrayOutputStream.toByteArray()), Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
}

Mat to BufferedImage

public static BufferedImage Mat2BufferedImage(Mat matrix)throws IOException {
    MatOfByte mob=new MatOfByte();
    Imgcodecs.imencode(".jpg", matrix, mob);
    return ImageIO.read(new ByteArrayInputStream(mob.toArray()));
}

Note, Though it's very negligible. However, in this way, you can get a reliable solution but it uses encoding + decoding. So you lose some performance. It's generally 10 to 20 milliseconds. JPG encoding loses some image quality also it's slow (may take 10 to 20ms). BMP is lossless and fast (1 or 2 ms) but requires little more memory (negligible). PNG is lossless but a little more time to encode than BMP. Using BMP should fit the most cases I think.

Nice, thank you. This solution works independent of the image input format of BufferedImage. It works for 24bit JPG and at the same time also for 32bit PNG. — asmaier, Feb 20 '18 at 18:13
this saved my butt, I was getting really weird color distortion until I came across this method, which was weird cause if I saved the image then loaded it with imread it had worked just fine, but I had no reason to be saving those images. — thekevshow, Sep 09 '18 at 03:58

score 25 · Answer 3 · answered Dec 15 '15 at 15:39

25

This one worked fine for me, and it takes from 0 to 1 ms to be performed.

public static Mat bufferedImageToMat(BufferedImage bi) {
  Mat mat = new Mat(bi.getHeight(), bi.getWidth(), CvType.CV_8UC3);
  byte[] data = ((DataBufferByte) bi.getRaster().getDataBuffer()).getData();
  mat.put(0, 0, data);
  return mat;
}

answered Dec 15 '15 at 15:39

Jorge Mardones

251
3
4

Thank you. Above functionality worked for me with a little change. I am dealing with gray scale single channel images. I used `Mat mat = new Mat(bi.getHeight(), bi.getWidth(), CvType.CV_8UC1);` and it worked for me. – Karthik N G Dec 20 '16 at 15:38
3

How will you do for RGBA images? – arqam May 25 '17 at 13:41

score 4 · Answer 4 · answered Feb 03 '17 at 16:15

I use following code in my program.

protected Mat img2Mat(BufferedImage in) {
        Mat out;
        byte[] data;
        int r, g, b;

        if (in.getType() == BufferedImage.TYPE_INT_RGB) {
            out = new Mat(in.getHeight(), in.getWidth(), CvType.CV_8UC3);
            data = new byte[in.getWidth() * in.getHeight() * (int) out.elemSize()];
            int[] dataBuff = in.getRGB(0, 0, in.getWidth(), in.getHeight(), null, 0, in.getWidth());
            for (int i = 0; i < dataBuff.length; i++) {
                data[i * 3] = (byte) ((dataBuff[i] >> 0) & 0xFF);
                data[i * 3 + 1] = (byte) ((dataBuff[i] >> 8) & 0xFF);
                data[i * 3 + 2] = (byte) ((dataBuff[i] >> 16) & 0xFF);
            }
        } else {
            out = new Mat(in.getHeight(), in.getWidth(), CvType.CV_8UC1);
            data = new byte[in.getWidth() * in.getHeight() * (int) out.elemSize()];
            int[] dataBuff = in.getRGB(0, 0, in.getWidth(), in.getHeight(), null, 0, in.getWidth());
            for (int i = 0; i < dataBuff.length; i++) {
                r = (byte) ((dataBuff[i] >> 0) & 0xFF);
                g = (byte) ((dataBuff[i] >> 8) & 0xFF);
                b = (byte) ((dataBuff[i] >> 16) & 0xFF);
                data[i] = (byte) ((0.21 * r) + (0.71 * g) + (0.07 * b));
            }
        }
        out.put(0, 0, data);
        return out;
    }

Reference: here

score 3 · Answer 5 · answered Mar 11 '13 at 12:56

3

I found a solution here. The solution is similar to Andriys.

Camera c;
c.Connect();
c.StartCapture();
Image f2Img, cf2Img;
c.RetrieveBuffer(&f2Img);
f2Img.Convert( FlyCapture2::PIXEL_FORMAT_BGR, &cf2Img );
unsigned int rowBytes = (double)cf2Img.GetReceivedDataSize()/(double)cf2Img.GetRows();

cv::Mat opencvImg = cv::Mat( cf2Img.GetRows(), cf2Img.GetCols(), CV_8UC3, cf2Img.GetData(),rowBytes );

answered Mar 11 '13 at 12:56

Jompa234

1,228
2
14
24

That would require JNI. – AMDG Feb 21 '15 at 22:19

score 3 · Answer 6 · answered Jan 11 '20 at 12:48

To convert from BufferedImage to Mat I use the method below:

    public static Mat img2Mat(BufferedImage image) {
        image = convertTo3ByteBGRType(image);
        byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
        Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
        mat.put(0, 0, data);
        return mat;
    }

Before converting into Mat, I change the type of bufferedImage to TYPE_3BYTE_BGR, because to some types BufferedImages the method ((DataBufferByte) image.getRaster().getDataBuffer()).getData(); may return int[] and that would break the code.

Below is the method for converting to TYPE_3BYTE_BGR.

    private static BufferedImage convertTo3ByteBGRType(BufferedImage image) {
        BufferedImage convertedImage = new BufferedImage(image.getWidth(), image.getHeight(),
                BufferedImage.TYPE_3BYTE_BGR);
        convertedImage.getGraphics().drawImage(image, 0, 0, null);
        return convertedImage;
    }

This is the method that will properly work everytime and is quite fast as well. The other answer lack details or break depending on the input image. It would be even a tiny bit better if the intermediate Graphics2D object used to paint the image were disposed (using dispose()). — Christoph Henkelmann, Sep 29 '21 at 17:25

score 2 · Answer 7 · answered Jul 13 '20 at 14:53

When you use as JavaCP wrapper bytedeco library (version 1.5.3) then you can use Java2DFrameUtils.

Simple usage is:

import org.bytedeco.javacv.Java2DFrameUtils;
...
BufferedImage img = ImageIO.read(new File("some/image.jpg");
Mat mat = Java2DFrameUtils.toMat(img);

Note: don't mix different wrappers, bytedeco Mat is different than opencv Mat.

score 1 · Answer 8 · answered Feb 19 '13 at 13:55

1

One simple way would be to create a new using

Mat newMat = Mat(rows, cols, type);

then get the pixel values from your BufferedImage and put into newMat using

newMat.put(row, col, pixel);

answered Feb 19 '13 at 13:55

Ted W.

220
2
9

score -6 · Answer 9 · edited Sep 20 '14 at 19:08

-6

You can do it in OpenCV as follows:

File f4 = new File("aa.png");
Mat mat = Highgui.imread(f4.getAbsolutePath());

edited Sep 20 '14 at 19:08

Lucas Zamboulis

2,494
5
24
27

answered Sep 20 '14 at 18:44

Ishan Randeniya

7
1

Converting `BufferedImage` to `Mat` in OpenCV

9 Answers9

Linked

Related