I need to send an image file to Clarifai server for image classification, but I keep getting "skipped frames, application doing too much work"

Question

I'm using a new thread to send the file, and a bit of code snippet to convert the bitmap to a file. The converting from bitmap to file is REALLY slow, and it seems like the sending of information to clarifai doesn't do anything...

//Convert bitmap to byte array
            Bitmap bitmap = mResultsBitmap;
            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            bitmap.compress(Bitmap.CompressFormat.PNG, 0 /*ignored for PNG*/, bos);
            byte[] bitmapdata = bos.toByteArray();

//write the bytes in file
            FileOutputStream fos = null;
            try {
                fos = new FileOutputStream(f);
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            }
            try {
                fos.write(bitmapdata);
            } catch (IOException e) {
                e.printStackTrace();
            }
            try {
                fos.flush();
            } catch (IOException e) {
                e.printStackTrace();
            }
            try {
                fos.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            final File pFIle2 = f;
            //TODO: clarifai stuff
            //TODO: clarifai stuff
            Log.e("this:"," this is running 0");
            Thread thread = new Thread(new Runnable() {
                @Override
                public void run() {
                    Log.e("this:", " this is running 1");
                    client = new ClarifaiBuilder("mykeyhere1234}").buildSync();
                    Log.e("this:", " this is running 2");
                    Thread th = new Thread(new Runnable() {
                        @Override
                        public void run() {
                            Log.e("this:", " this is running 3");
                            Log.e("this", client.getDefaultModels().generalModel().predict()
                                    .withInputs(
                                            ClarifaiInput.forImage(ClarifaiImage.of(pFIle2))
                                    )
                                    .executeSync().rawBody());
                            Log.e("this:", " this is running 4");
                        }
                    });


                }
            });

This bit of code snippet is in the onActivityResult method. None of the log messages are printing except for "0"

Answer 1

The error in your log message is thrown when the API client is unable to acquire complete default models (i.e. general model and others) from the server after several repeated attempts. Please make sure:

• your API key is valid and has permissions to do predict,
• your app is able connect to the Clarifai server.

Two additional points:

• Is this code running in a loop somewhere? The new ClarifaiBuilder/ClarifaiClient instance should be constructed only once (probably at the start of the application).
• You can perhaps simplify predicting a local file. Please see this example.

Answer 2

First of all, you should put the bitmap conversion into the Thread also, so that it is run asynchronously. This will prevent the stutters and skipped frames message.

The reason why the Clarifai service does not appear to be working is because you did not start() the Thread. Additionally, there is no reason to put a Thread inside another Thread.

All together, here is the fixed code:

final Bitmap bitmap = mResultsBitmap;

final File pFile2 = f;

ClarifaiClient client = new ClarifaiBuilder("mykeyhere1234}").buildSync();

DefaultModels defaultModels = client.getDefaultModels();

Log.e("this:"," this is running 0");
Thread thread = new Thread(new Runnable() {
    @Override
    public void run() {
        //Convert bitmap to byte array
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        bitmap.compress(Bitmap.CompressFormat.PNG, 0 /*ignored for PNG*/, bos);
        byte[] bitmapdata = bos.toByteArray();

        //write the bytes in file
        FileOutputStream fos = null;
        try {
            fos = new FileOutputStream(pFile2);
            fos.write(bitmapdata);
        } catch (FileNotFoundException e | IOException e) {
            e.printStackTrace();
        } finally {
            try {
                fos.flush();
                fos.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        //TODO: clarifai stuff
        //TODO: clarifai stuff
        Log.e("this:", " this is running 1");
        client = 
        Log.e("this:", " this is running 2");
        Log.e("this:", " this is running 3");
        Log.e("this", models.generalModel().predict()
                .withInputs(
                        ClarifaiInput.forImage(ClarifaiImage.of(pFile2))
                )
                .executeSync().rawBody());
        Log.e("this:", " this is running 4");

     }
});

thread.start();

Answer 3

Is is always a best practice to keep the bitmap related operations on background thread, try this you will not get the log related to skipped frames, doing too much work on main thread. use async task to perform the bitmap operations.