Android MLKit - Internal error has occurred when executing Firebase ML tasks

Question

Hi I have a custom model that im using in an android app, how ever when i try to run it, an MLkitExceptions is thrown, the log output for said error is the following:

Internal error has occurred when executing Firebase ML tasks

My java code for my app looks like the following:

FirebaseLocalModel localModel = new FirebaseLocalModel.Builder("local_places_recommend")
                .setAssetFilePath("recsys.tflite").build();

FirebaseModelManager.getInstance().registerLocalModel(localModel);



FirebaseModelOptions options = new FirebaseModelOptions.Builder()
                .setLocalModelName("local_places_recommend")
                .build();

FirebaseModelInterpreter firebaseInterpreter = FirebaseModelInterpreter.getInstance(options);

FirebaseModelInputOutputOptions inputOutputOptions =
       new FirebaseModelInputOutputOptions.Builder()
                .setInputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{1, 3})
                .setOutputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{1, 1, 34})
                .build();

float[][] input = new float[1][3];

input[0][0] = d1;
input[0][1] = d2;
input[0][2] = d3;


FirebaseModelInputs inputs = new FirebaseModelInputs.Builder()
                .add(input)
                .build();

whenever i try to run the FirebaseModelInterpreter I get that error and nothing else.

The shape of the input tensor and output tensor are the following respectively:

[1 3]
<class 'numpy.float32'>
[ 1  1 34]
<class 'numpy.int64'>

Any help or insight about this issue would be gladly appreciated.

Answer 1

If You are working with CameraX you may get this error.

I/CameraXBasic: Internal error has occurred when executing ML Kit tasks 

Please ensure that you close the ImageProxy and image in analyze function after the MLKit operation

Heere's small snippet

override fun analyze(imageProxy: ImageProxy) {

     val image:Image? = imageProxy.image

     image?.let {img->
               val iImage = InputImage
                 .fromMediaImage(img,imageProxy.imageInfo.rotationDegrees)
                textRecognizer.process(iImage)
                    .addOnSuccessListener { listener(it)}
                    .addOnFailureListener { Log.i(TAG,it.toString()) }
                    .addOnCompleteListener {
                        image.close()
                        imageProxy.close() }
     }

}

Answer 2

As mentioned in the documentation of Firebase : https://firebase.google.com/docs/reference/android/com/google/firebase/ml/custom/FirebaseModelDataType

There is no type INT64, please try to write INT32 instead of FLOAT32.

FirebaseModelInputOutputOptions inputOutputOptions =
           new FirebaseModelInputOutputOptions.Builder()
                    .setInputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{1, 3})
                    .setOutputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{1, 1, 34})// try to change FLOAT32 to Int32
                    .build();

Answer 3

I had got the same problem because my version control changed my tflite Model and did not update my code. I had the wrong input shape for my model. I only changed:

val inputOutputOptions = FirebaseModelInputOutputOptions.Builder()
                .setInputFormat(0, FirebaseModelDataType.FLOAT32, intArrayOf(1, **RightSize, RightSize**, 3))
                .setOutputFormat(0, FirebaseModelDataType.FLOAT32, intArrayOf(1, 4))
                .build()

Source: https://github.com/tensorflow/tensorflow/issues/30187

Answer 4

1. You need to add above callback for cameraX

2. Update feature dependency to run the app by required check for below hardware feature.