Getting Mats from frames in a gif using OpenCV and Java

Question

I am trying to get frames from a gif using OpenCV. I found Convert each animated GIF frame to a separate BufferedImage and used the second suggestion. I modified it slightly to return an array of Mats instead of BufferedImages.

I tried two methods to get bufferedImages from the gif. Each presented different problems.

With the previous thread's suggestion
```
BufferedImage fImage=ir.read(i);
```
The program calls a "ArrayIndexOutOfBoundsException: 4096"

With the original code from the previous thread.

BufferedImage fImage=ir.getRawImageType(i).createBufferedImage(ir.getWidth(i),ir.getHeight(i));

Each frame is a monotone color(not all black though) and the mat derived from the BufferedImage is empty.

System.loadLibrary( Core.NATIVE_LIBRARY_NAME );
ArrayList<Mat> frames = new ArrayList<Mat>();
ImageReader ir = new GIFImageReader(new GIFImageReaderSpi());
ir.setInput(ImageIO.createImageInputStream(new File("ronPaulTestImage.gif")));

for(int i = 0; i < ir.getNumImages(true); i++){
    BufferedImage fImage=ir.read(i);
    //BufferedImage fImage=ir.getRawImageType(i).createBufferedImage(ir.getWidth(i), ir.getHeight(i));

    fImage = toBufferedImageOfType(fImage, BufferedImage.TYPE_3BYTE_BGR);
    //byte[] pixels = ((DataBufferByte) r.getRaster().getDataBuffer()).getData();
    Mat m=new Mat();
    //m.put(0,0,pixels);
    m.put(0, 0,((DataBufferByte) fImage.getRaster().getDataBuffer()).getData());

    if(i==40){
    //a test, writes the mat and the image at the specified frame to files, exits
        ImageIO.write(fImage,"jpg",new File("TestError.jpg"));
        Imgcodecs.imwrite("TestErrorMat.jpg",m);
        System.exit(0);
}

Here is the gif I used

score 3 · Answer 1 · answered May 25 '15 at 10:55

Following Spektre's advice I found a better gif which fixed the monochromatic bufferedImages. The lack of viewable Mats was caused by my usage of the default constructor when declaring the Mat.

Working Code

    public static ArrayList<Mat> getFrames(File gif) throws IOException{
    ArrayList<Mat> frames = new ArrayList<Mat>();
    ImageReader ir = new GIFImageReader(new GIFImageReaderSpi());
    ir.setInput(ImageIO.createImageInputStream(gif));
    for(int i = 0; i < ir.getNumImages(true); i++){
        BufferedImage fImage=ir.read(i);
        fImage = toBufferedImageOfType(fImage, BufferedImage.TYPE_3BYTE_BGR);
        byte[] pixels = ((DataBufferByte) fImage.getRaster().getDataBuffer()).getData();
        Mat m=new Mat(fImage.getHeight(), fImage.getWidth(), CvType.CV_8UC3);
        m.put(0,0,pixels);
        if(i==15){//a test, writes the mat and the image at the specified frame to files, exits
            ImageIO.write(fImage,"jpg",new File("TestError.jpg"));
            Imgcodecs.imwrite("TestErrorMat.jpg",m);
            System.exit(0);
        }
        frames.add(m);
        }
    return frames;
}

heh didn't see your answer until now. I had my fair share of problems with default constructor myself (in C++ and some nasty compiler bug) which leads to this: [bds 2006 C hidden memory manager conflicts](http://stackoverflow.com/a/18016392/2521214) solution may be interest you too (even if you are using JAVA) — Spektre, Jan 31 '17 at 08:37

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

1

I am not using libs for gif nor Java nor OpenCV but the ArrayIndexOutOfBoundsException: 4096

means that the dictionary is not cleared properly. The gif of yours is buggy I tested it and it contains errors not enough clear codes are present for some frames. If your GIF decoder does not check/handle such case then it simply crash because its dictionary growth more then GIF limit 4096/12bit

Try another GIF not some buggy ones ...

have tested your gif and it has around 7 clear codes per frame and contains 941 errors in total (absence of clear code resulting in dictionary overrun)

If you have source code for the GIF decoder

then just find part of decoder where new item is added to dictionary and add

if (dictionary_items<4096)

before it ... If you ignore the wrong entries the image looks still OK most likely the encoder in which this was created was not properly coded.

edited Jun 20 '20 at 09:12

Community

1
1

answered May 25 '15 at 07:16

Spektre

49,595
11
110
380

So I found the [patched version](http://pastebin.com/h58zjT8K) of the Gif decoder and it kinda works with the broken gifs. Although it seems to decode the gifs properly(The bufferedImages and Mat taken from each frame look fine),when the frames are recompiled into gifs, the recompiled Gifs from these frames are just fuzzy static. Note that this does not happen with gifs that were unaffected by the bug before(they work as they should). – lleontan May 25 '15 at 19:23
@lleontan that is odd most likely the patch is not good enough and the decoder overwrites something in memory that it should not to check for that extract the frames save them somewhere, restart the application and then recompile the images back to gif (without decoding gif this time) if the problem is still there then it is something else ... if the problem disappears then pached version is still leaking memory or overrun/overwrites something (just not throwing exceptions)... – Spektre May 25 '15 at 19:35
I did some tests and I isolated the problem to my mat->BufferedImage function. Its all done now! Thank you so much for the help! – lleontan May 25 '15 at 23:29

Eric · Answer 3 · 2021-09-22T02:12:49.573

with bytedeco opencv, simple way to get the first frame of gif:

import java.awt.image.{BufferedImage, DataBufferByte}
import java.io.ByteArrayInputStream
import com.sun.imageio.plugins.gif.{GIFImageReader, GIFImageReaderSpi}

import javax.imageio.ImageIO
import javax.swing.JFrame
import org.bytedeco.javacv.{CanvasFrame, OpenCVFrameConverter}
import org.bytedeco.opencv.opencv_core.Mat
import org.opencv.core.CvType

def toMat(bi: BufferedImage): Mat = {
  val convertedImage = new BufferedImage(bi.getWidth(), bi.getHeight(), BufferedImage.TYPE_3BYTE_BGR)
  val graphics = convertedImage.getGraphics
  graphics.drawImage(bi, 0, 0, null)
  graphics.dispose()

  val data = convertedImage.getRaster.getDataBuffer.asInstanceOf[DataBufferByte].getData
  val mat = new Mat(convertedImage.getHeight, convertedImage.getWidth, CvType.CV_8UC3)
  mat.data.put(data: _*)

  mat
}

def show(image: Mat, title: String): Unit = {
  val canvas = new CanvasFrame(title, 1)
  canvas.setDefaultCloseOperation(javax.swing.WindowConstants.EXIT_ON_CLOSE)
  canvas.showImage(new OpenCVFrameConverter.ToMat().convert(image))
}


val imageByteArrayOfGif = Array(....) // can be downloaded with java.net.HttpURLConnection

val ir = new GIFImageReader(new GIFImageReaderSpi())
val in = new ByteArrayInputStream(imageByteArray)
ir.setInput(ImageIO.createImageInputStream(in))

val bi = ir.read(0) // first frame of gif
val mat = toMat(bi)

show(mat, "buffered2mat")

Getting Mats from frames in a gif using OpenCV and Java

3 Answers3

Linked