0

I am working with the Camera2 basic sample. And I wish to do some basic image processing with openCV.

This question discusses the same problem however the answers only talk about acquiring preview frames via ImageReader.

My question is
Is it possible to process the image frames before they are rendered/displayed to the SurfaceTexture.

vyi
  • 1,078
  • 1
  • 19
  • 46

1 Answers1

2

An ImageReader gives you a set of ByteBuffers in each Image you acquire from it; you can operate on those either in Java or in native code.

The simplest case is capturing a JPEG and just saving it to disk, but you can also request YUV_420_888 data and then process it however you want.

Edit in response to comment:

If you've gotten a SurfaceTexture from a TextureView, and passed it to the camera, then you can't intercept the buffers in between. If you want to modify them, then you need to create an intermediate target that the camera sends buffers to, edit them, and then send them for display to the TextureView.

There are several options for that. Possibly the most efficient is using EGL in the middle:

Camera -> SurfaceTexture -> EGL -> SurfaceTexture -> TextureView

This requires a lot of boilerplate code to create the EGL context, but works well if your edits can be written as an EGL shader. You can render to the SurfaceTexture given by the TextureView by creating an EGLImage from it, if I recall correctly, and then you can create another SurfaceTexture that you pass to camera, which you use in the EGL shader as a texture to render from.

I'd recommend finding EGL tutorials since this requires quite a bit of code.

Eddy Talvala
  • 17,243
  • 2
  • 42
  • 47
  • Hi, Thanks for answering. I think you are confused. This question is in context of SurfaceTexture, **can we modify image before displaying** in a preview application (like the Camera2 basic sample application)? – vyi Aug 21 '18 at 05:42
  • Thanks for the clear response. If you know example basic EGL implementation do let me know. – vyi Aug 23 '18 at 05:03