4

I want to extend an app from Camera1 to Camera2 depending on the API. One core mechanism of the app consists in taking preview pictures at a rate of about 20 pics per second. With Camera1 I realized that by creating a SurfaceView, adding a Callback on its holder and after creation of the surface accessing the preview pics via periodic setOneShotPreviewCallbacks. That was pretty easy and reliable.

Now, when studying Camera2, I came "from the end" and managed to convert YUV420_888 to Bitmap (see YUV420_888 to Bitmap Conversion ). However I am struggling now with the "capture technique". From the Google example I see that you need to make a "setRepeating" CaptureRequest with CameraDevice.TEMPLATE_PREVIEW for displaying the preview e.g. on a surface view. That is fine. However, in order to take an actual picture I need to make another capture request with (this time) builder.addTarget(imageReader.getSurface()). I.e. data will be available within the onImageAvailable method of the imageReader.

The problem: the creation of the captureRequest is a rather heavy operation taking about 200ms on my device. Therefore, the usage of a capture request (whether with Template STILL_CAPTUR nor PREVIEW) can impossibly be a feasible approach for capturing 20 images per second, as I need it. The proposals I found here on SO are primarily based on the (educationally moderately efficient) Google example, which I don't really understand...

I feel the solution must be to feed the ImageReader with a contiuous stream of preview pics, which can be picked from there in a given frequency. Can someone please give some guidance on how to implement this? Many thanks.

Community
  • 1
  • 1
Settembrini
  • 1,366
  • 3
  • 20
  • 32

1 Answers1

4

If you want to send a buffer to both the preview SurfaceView and to your YUV ImageReader for every frame, simply add both Surfaces to the repeating preview request as targets.

Generally, a capture request can target any subset (or all) of the session's configured output targets.

Also, if you do want to only capture an occasional frame to your YUV ImageReader with .capture(), you don't have to recreate the capture request builder each time; just call .build() again on the same builder, or just reuse the actual constructed CaptureRequest if you're not changing any settings.

Even with this occasional capture, you probably want to include the preview Surface as a target in the YUV capture request, so that there's no skipped frame in the displayed preview.

Eddy Talvala
  • 17,243
  • 2
  • 42
  • 47
  • Thank you Eddy, this is very helpful, and sounds like an intuitive approach. However, in order to implement it efficiently I see the need to study Camera2 in more detail, as there are still many aspects I don't understand (e.g. the role of threading). I will come back here, accept your answer and hopefully be able to post some code that helps other too. Maybe I will have some follow-up questions until then... Thanks again. – Settembrini Mar 18 '16 at 16:59
  • 3
    I am close to give it up. Everytime I add the imageReader as an additional target either the preview is not visible anymore or I get a crash "cannot use a surface that wasn't configured". Also I searched for many hours for an complete example of getting preview pictures via imagereader (a rather basic application I think), but I didn't find any. And I tried to simplify the Google example to the extent possible, but it is still too complex to assess the relevance of the individual parts for my problem.... – Settembrini Mar 18 '16 at 22:03
  • 2
    Sorry to hear that. "cannot use a surface that wasn't configured" means that you didn't include the ImageReader Surface as part of the CameraDevice.createCaptureSession arguments. You need both Surfaces as the argument for createCaptureSession, and then for the CaptureRequest you submit with setRepeatingRequest. One important detail: Make sure to close Images received from the ImageReader as soon as you're done with them; otherwise everything stalls (there's a fixed number of buffers in the Reader, and the camera stops if it can't get an empty one to fill). – Eddy Talvala Mar 22 '16 at 19:50
  • 2
    That was the missing link! The only problem was indeed that I didn't add the ImageReaders' surface to the Arrays.asList(...) argument of the mCameraDevice.createCaptureSession().... So easy. Thank you, Eddy! Now I am really looking forward working with Camera2! – Settembrini Mar 23 '16 at 21:30