I need to normalize a bitmap and store it in a TensorImage. Is there any way to do this?

Question

Bitmap holds only integer values (0-255). I need to divide each pixel value by 255. The bitmap is converted to a TensorImage and then getBuffer() is called when passing it to the interpreter that predicts output.(tflite.run()) Somewhere in the middle, I have to divide each RGB pixel by 255. I'm afraid there is another drawback as the getBuffer() function returns a byte buffer. I'm not able to find much documentation on TensorFlow lite functions. So I am unsure if tflite.run() can accept only byte buffers or not. I am coding in Java and am new to Android AppD. Please help as this normalization is essential to predict the right value.

Here is the code that converts the bitmap to tensorimage after resizing. It is here I need to divide each pixel value by 255 but am stumped.

 private TensorImage resizePic(Bitmap bp) {
        ImageProcessor imageProcessor =
                new ImageProcessor.Builder()
                        .add(new ResizeOp(60, 60, ResizeOp.ResizeMethod.BILINEAR))
                        .build();
        TensorImage tImage = new TensorImage(DataType.FLOAT32);
        tImage.load(bp);
        tImage = imageProcessor.process(tImage);
        return tImage;
    }

Here is the line that runs the model

tflite.run(tImage.getBuffer(), probabilityBuffer.getBuffer());

probabilityBuffer holds the output.

There are also other operators inside ImageProcessor class https://www.tensorflow.org/lite/inference_with_metadata/lite_support#imageprocessor_architecture you can use the NormalizeOp. — Farmaker, Nov 01 '21 at 05:00

Rukia394 · Accepted Answer · 2020-06-14T12:43:44.483

I was able to construct suitable functions using the following links-

The second link is in Kotlin. Here is the code:

private ByteBuffer convertBitmapToByteBuffer(Bitmap bp) {
        ByteBuffer imgData = ByteBuffer.allocateDirect(Float.BYTES*60*60*3);
        imgData.order(ByteOrder.nativeOrder());
        Bitmap bitmap = Bitmap.createScaledBitmap(bp,60,60,true);
        int [] intValues = new int[60*60];
        bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());

        // Convert the image to floating point.
        int pixel = 0;

        for (int i = 0; i < 60; ++i) {
            for (int j = 0; j < 60; ++j) {
                final int val = intValues[pixel++];

                imgData.putFloat(((val>> 16) & 0xFF) / 255.f);
                imgData.putFloat(((val>> 8) & 0xFF) / 255.f);
                imgData.putFloat((val & 0xFF) / 255.f);
            }
        }
        return imgData;
    }

Here, 60 is my required input image height and width. Also, this method doesn't require use of a TensorImage. So the final call of tflite.run() looks like this:

tflite.run(convertBitmapToByteBuffer(bp), probabilityBuffer.getBuffer());

Here, bp is the bitmap image.

score 0 · Answer 2 · answered Mar 23 '21 at 14:07

0

When you are training model don't normalise image. So when you deploy your application there is no need to normalise bitmap image.

answered Mar 23 '21 at 14:07

Ankit

1

score 0 · Answer 3 · answered May 22 '21 at 17:58

Your first reference gave an example of using Opencv to do the conversion. Here's what I came up with that's working:

private ByteBuffer getImageDataForTfliteModelOpencv(Bitmap input) {
    if (input == null) {
        return null;
    }
    // Allocate output ByteBuffer
    ByteBuffer output = ByteBuffer.allocateDirect(1 * TFL_IMAGE_SIZE *
            TFL_IMAGE_SIZE * 3 * Float.BYTES);
    //
    output.order(ByteOrder.nativeOrder());
    output.rewind();

    Mat bufmat = new Mat();
    Mat newmat = new Mat(TFL_IMAGE_SIZE, TFL_IMAGE_SIZE, CvType.CV_32FC3);

    Utils.bitmapToMat(input, bufmat);
    Imgproc.cvtColor(bufmat, bufmat, Imgproc.COLOR_RGBA2RGB);
    bufmat.convertTo(newmat, CvType.CV_32FC3, 1.0/255.0);
    //
    // Write the image float data to the output ByteBuffer
    float buf[] = new float[TFL_IMAGE_SIZE * TFL_IMAGE_SIZE * 3];
    newmat.get(0,0, buf); // Get the float data
    output.asFloatBuffer().put(buf); // Write it as a stream of bytes
    return output;
}

The returned ByteBuffer can then be easily loaded into a TensorBuffer. I tested both methods and this Opencv method is about 50msec faster for a 112x112 image.

score 0 · Answer 4 · edited Oct 31 '21 at 19:33

As mentioned here use the below code from here for converting Bitmap to ByteBuffer(float32)

private ByteBuffer convertBitmapToByteBuffer(Bitmap bitmap) {
    ByteBuffer byteBuffer = ByteBuffer.allocateDirect(4 * BATCH_SIZE * inputSize * inputSize * PIXEL_SIZE);
    byteBuffer.order(ByteOrder.nativeOrder());
    int[] intValues = new int[inputSize * inputSize];
    bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());
    int pixel = 0;
    for (int i = 0; i < inputSize; ++i) {
        for (int j = 0; j < inputSize; ++j) {
            final int val = intValues[pixel++];
            byteBuffer.putFloat((((val >> 16) & 0xFF)-IMAGE_MEAN)/IMAGE_STD);
            byteBuffer.putFloat((((val >> 8) & 0xFF)-IMAGE_MEAN)/IMAGE_STD);
            byteBuffer.putFloat((((val) & 0xFF)-IMAGE_MEAN)/IMAGE_STD);
        }
    }
    return byteBuffer;
}

score 0 · Answer 5 · answered Jan 04 '23 at 15:43

I found the answer：

private TensorImage resizePic(Bitmap bp) {
    ImageProcessor imageProcessor =
            new ImageProcessor.Builder()
                    .add(new ResizeOp(60, 60, ResizeOp.ResizeMethod.BILINEAR))
                    .add(new NormalizeOp(0f, 255f))
                    .build();
    TensorImage tImage = new TensorImage(DataType.FLOAT32);
    tImage.load(sourceBitmap);
    System.out.println("tensorImage0: " + tImage.getTensorBuffer().getFloatArray()[0]);
    tImage = imageProcessor.process(tImage);
    System.out.println("tensorImage1: " + tImage.getTensorBuffer().getFloatArray()[0]);
    return tImage;
}

terminal:

System.out: tensorImage0: 232.0
System.out: tensorImage1: 0.9254902

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). — Kunal Varpe, Jan 04 '23 at 15:54

I need to normalize a bitmap and store it in a TensorImage. Is there any way to do this?

5 Answers5