Display OpenCV Mat with JavaFX

Question

I would like to display Mat objects from OpenCV directly with JavaFX. I have seen that it is possible to convert a Mat object into a BufferedImage. But as far as I know you can't display a BufferedImage with JavaFX, so another conversion would have to take place.

Is there a way to directly convert it into a data structure that is displayable by JavaFX?

It's probably possible to convert directly to FX, using a `WritableImage`. Either by copying pixels from the `Mat` to the `WritableImage`'s `PixelWriter`, or by implementing a `PixelReader` backed by the `Mat`, and use that for instantiating the `WritableImage` directly. — Harald K, Jan 05 '15 at 12:09

score 7 · Accepted Answer · answered Jan 12 '15 at 21:11

7

I have found a direct way to convert a Mat object to a JavaFX Image object.

MatOfByte byteMat = new MatOfByte();
Highgui.imencode(".bmp", mat, byteMat);
return new Image(new ByteArrayInputStream(byteMat.toArray()));

You can also encode it to .jpg but .bmp is faster.

answered Jan 12 '15 at 21:11

TomTom

2,820
4
28
46

Why my OpenCV doesn't come with HighGUI library/class ? – Zeta.Investigator Mar 29 '16 at 12:42

deb0ch · Answer 2 · 2015-11-19T16:09:07.090

4

TomTom's answer was very helpful and solved the problem, but Highgui does no longer have any bindings in java.

As of OpenCV 3.0.0, the up-to-date code is more like:

MatOfByte byteMat = new MatOfByte();
Imgcodecs.imencode(".bmp", mat, byteMat);
return new Image(new ByteArrayInputStream(byteMat.toArray()));

edited Nov 19 '15 at 16:09

answered Nov 09 '15 at 08:28

deb0ch

1,133
2
16
25

Your answer is the solution for OpenCV version 3. Earlier versions use Highgui. – Roland Nov 17 '15 at 06:06
Thank you for your precision, I just edited and corrected my answer. – deb0ch Nov 19 '15 at 16:10

score 2 · Answer 3 · answered Jan 03 '15 at 13:19

The silly way to do this is to convert the Mat to a BufferedImage, and then that to an Image so that it can be displayed inside an ImageView:

Mat >> BufferedImage >> Image >> ImageView

Assuming you know how to do the 1st conversion, the rest would be something like this:

import javafx.embed.swing.SwingFXUtils;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;

Image image = SwingFXUtils.toFXImage(bufImage, null);

ImageView imgView = new ImageView();
imgView.setImage(image);

I haven't tested the code, but that's the general idea.

score 1 · Answer 4 · answered Sep 19 '16 at 19:12

Instead of encoding and decoding to get the byte array you need, you can just declare an array and use the get method. This is Scala code, but hopefully it's intelligible:

val arr = new Array[Byte](w * h * 3)
mat.get(0, 0, arr)
pw.setPixels(0, 0, w, h, PixelFormat.getByteRgbInstance, arr, 0, w * 3)

This code declares the array arr of type byte[] with the size corresponding to an image of h by w and 3 channels. Then the data is copied from the mat object into the array, and passed to the setPixels method from the PixelWriter object pw.

Display OpenCV Mat with JavaFX

4 Answers4

Linked