1

I have a camera sending frames to a SurfaceView. I would like to get these frames out of the surface view and send them elsewhere. In their final form, the images must be in JPEG format. To accomplish this currently, I am creating a YUV image from the byte[] and then calling compressToJpeg. However, when I invoke compressToJpeg on every frame rather than doing nothing but displaying it, my FPS goes from ~30 to ~4. I commented out the other lines and this function appears to be the culprit.

public void onNewRawImage(byte[] data, Size size) {
    // Convert to JPG
    YuvImage yuvimage=new YuvImage(data, 
        ImageFormat.NV21, size.width, size.height, null);

    ByteArrayOutputStream baos = new ByteArrayOutputStream();

    yuvimage.compressToJpeg(new Rect(0, 0, yuvimage.getWidth(), 
        yuvimage.getHeight()), 80, baos);

    byte[] jdata = baos.toByteArray();

    // Convert to Bitmap
    Bitmap bitmap = BitmapFactory.decodeByteArray(jdata, 0, jdata.length); 
}

Is it possible to start in the JPEG format rather than having to convert to it? I am hoping I am making a mistake somewhere. Any help is greatly appreciated, thank you.

unknown
  • 55
  • 8
  • "Is it possible to start in the JPEG format rather than having to convert to it?" -- I don't think so. "I am hoping I am making a mistake somewhere" -- are you doing this work as part of the preview frame callback? Or are you doing the work asynchronously? – CommonsWare Dec 03 '15 at 23:21
  • 1
    This function is really not friendly. You're doing a lot of byte mainpulation, including compressing the data to a jpeg then reinflating it to a bitmap (which holds uncompressed data). Not doing a conversion then uncompression would save you a ton of time. – Gabe Sechan Dec 03 '15 at 23:25
  • Check out http://stackoverflow.com/questions/9325861/converting-yuv-rgbimage-processing-yuv-during-onpreviewframe-in-android I think this is more or less what you're asking – Gabe Sechan Dec 03 '15 at 23:27
  • @CommonsWare I am calling the onNewRawImage from the `onPreviewFrame(byte[] data, Camera camera)` function. – unknown Dec 03 '15 at 23:28
  • 1
    Well, I'd start then by not doing that. Set up multiple preview frame buffers, and process them asynchronously on your own worker thread (or thread pool). – CommonsWare Dec 03 '15 at 23:29
  • @CommonsWare I am not extremely familiar with multi-threaded programming. I will spend some time looking into this. – unknown Dec 03 '15 at 23:38
  • @GabeSechan The main reason why I am reinflating to a bitmap is because I am scaling a very large image down to a smaller resolution to reduce latency (I am sending the image to another machine in real time over wifi). If there is a way to scale the image without inflating to a Bitmap, we could skip this step, however, I am unsure of how to accomplish this without the Bitmap class. Do you know of a way to efficiently scale down a YUV image? – unknown Dec 03 '15 at 23:52
  • You can definitely manipulate the raw UAV bytes, and do so more effectively than using built in functions. Here's a link to some ways http://stackoverflow.com/questions/17187193/resize-downsize-yuv420sp-image and a library that does it for you but you'll have to use JNI to reach it https://code.google.com/p/libyuv/ – Gabe Sechan Dec 04 '15 at 00:04
  • 1
    You may want to look at RenderScript ScriptIntrinsics for YUV to RGB: http://stackoverflow.com/q/20358803/295004 and look at the RenderScript test case apps https://android.googlesource.com/platform/frameworks/rs/+/master/java/tests/ especially LivePreview – Morrison Chang Dec 04 '15 at 05:39
  • @CommonsWare I now have four preview frame buffers and my onPreviewFrame instantiates a thread like: `new Thread(new Runnable() {` to perform the operation asynchronously. I removed inflation to bitmap & scaling temporarily and with these changes my FPS has improved significantly. The only issue I have now is reducing latency. I need to scale the image in its YUV form so I can avoid the Bitmap inflation. I will look into the suggestions regarding this now, thanks all. – unknown Dec 04 '15 at 19:46

0 Answers0