Using OpenCV Mat for reading text using tesseract (Java)

Question

I need to read text using tesseract, for my processed image which is done using OpenCV. I have already tried the methods mentioned in the following link, but to no avail:

Method 1

Converting using MatOfBytes SO Link

public static PIX convertMatToPix(Mat mat) {

        LOGGER.info("Size of image: " + mat.size());
        MatOfByte bytes = new MatOfByte();
        Imgcodecs.imencode(".jpg", mat, bytes);
        ByteBuffer buff = ByteBuffer.wrap(bytes.toArray());
        LOGGER.info("Size for pixReadMem: " + new NativeSize(buff.capacity()).intValue());
        return pixReadMem(buff, new NativeSize(buff.capacity()).intValue());
    }

This method creates a white image out of my input image.

Method 2

Converting using Byte Array

byte[] imageBuffer = new byte[(int) (input.rows()*input.cols()*input.channels())];
        input.get(0, 0, imageBuffer);
PIX image = pixReadMem(imageBuffer, (int) (input.rows()*input.cols()*input.channels()));

This method fails to create an image, as I am outputting image using:

int writingFromPix = pixWriteAutoFormat("/home/abhishekkeshri/Pictures/ocr/pixOutput.jpg", image);
LOGGER.info("Writing from PIX returns: " + writingFromPix);

And it is returning 1, which signifies an error in writing image.

Method 3

Copying pixel-by-pixel value (SO link)

PIX mat8ToPix(Mat mat8)
    {
        PIX pixd = pixCreate(mat8.width(), mat8.height(), mat8.depth());
        for(int y=0; y<mat8.rows(); y++) {
            for(int x=0; x<mat8.cols(); x++) {
                double[] pixelDouble = mat8.get(y,x);
                int[] pixelInt = new int[pixelDouble.length];
                for(int i=0;i<pixelDouble.length;i++) {
                    pixelInt[i] = (int) pixelDouble[i];
                }
                pixSetPixel(pixd, y, x, pixelInt[0]);
            }
        }
        return pixd;
    }

The problem in this method is that get method returns an array of pixel, which depends on the channel. So just using the 0th value should be wrong. The logic seems correct but what am I doing wrong in this method?

Method 4

Directly using Mat to setImage in Tesseract

api.SetImage(imageBuffer, input.width(), input.height(), input.channels(), (int)input.step1());

This method also fails to get any output from Tesseract, as the output printed is blank.

Method 5

By simply reading the preprocessed image, and reading it using tesseract the output given is correct:

PIX image = pixRead("/home/abhishekkeshri/Pictures/ocr/clean.jpg");
api.SetImage(image);

This gives the correct result. But I can't use this as disk operations will be quite costly (writing output of opencv and saving it in a file, the opening that file using PIX)

Any help will be appreciated, as I am stuck in this problem since 2 days.

My maven dependencies:

    <dependency>
        <groupId>org.bytedeco.javacpp-presets</groupId>
        <artifactId>tesseract-platform</artifactId>
        <version>4.0.0-rc2-1.4.3</version>
    </dependency>
    <dependency>
        <groupId>org.openpnp</groupId>
        <artifactId>opencv</artifactId>
        <version>3.4.2-1</version>
    </dependency>
    <dependency>
        <groupId>net.sourceforge.lept4j</groupId>
        <artifactId>lept4j</artifactId>
        <version>1.10.0</version>
    </dependency>

score 0 · Answer 1 · answered May 30 '19 at 07:02

I have used below code for convert Mat to BufferImage.

Tesseract Code:

Tesseract tesseract1 = new Tesseract();
tesseract1.setDatapath(datapath);
tesseract1.doOCR(Mat2BufferedImage(mat);

Mat to BufferdImage Convert code:

static BufferedImage Mat2BufferedImage(Mat matrix) throws Exception {
    MatOfByte mob = new MatOfByte();
    Imgcodecs.imencode(".jpg", matrix, mob);
    byte ba[] = mob.toArray();

    BufferedImage bi = ImageIO.read(new ByteArrayInputStream(ba));
    return bi;
}

Using this code getting expected output.

Using OpenCV Mat for reading text using tesseract (Java)

1 Answers1