1

I am using OpenGLES2 output to display to a SurfaceView or encode to mp4 using MediaCodec.

However, I can only do one at a time. I can obviously draw using OpenGLES2 onto two separate surfaces but that would be a really inefficient use of the GPU.

What I want is to use some sort of reference counting to reuse the buffer to both draw on the screen and encode the single OpenGLES2 output. Like how camera service does in the Shared Surfaces concept.

Can can one do both display and encode of a buffer? Is there some sort of tee element (like in GStreamer) present in Android?

zeitgeist
  • 852
  • 12
  • 19

2 Answers2

1

There is no T-component available at the moment.

But you can avoid rendering twice by drawing to a Framebuffer Object and then copying the frame both to the screen and to the encoder.

Here's an example (pretty old).

dev.bmax
  • 8,998
  • 3
  • 30
  • 41
  • Thanks for the answer, what you have mentioned is drawing two times on different surfaces (which I already know), not like tee where we should be able to copy the finished buffer. Thus, I cannot accept your answer. – zeitgeist Sep 15 '21 at 04:19
  • Also, I was hoping for a refcounting solution so that there is no copy involved for greater speed and less memory usage. – zeitgeist Sep 15 '21 at 04:19
  • 1
    The FBO contains the finished frame, which is then copied to multiple surfaces. There are a couple of ways to copy the frame from the FBO (e.g. with `glBlitFramebuffer`). This is usually much faster than drawing the whole scene twice. – dev.bmax Sep 16 '21 at 17:31
  • https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glBlitFramebuffer.xhtml tells that `glBlitFramebuffer` is not available of OpenGLES2, do you have an alternative for OpenGLES2 android? – zeitgeist Sep 17 '21 at 03:54
  • 1
    First of all, I don't see a good reason not to require OpenGL ES 3 in 2021. An alternative is to draw a full screen textured quad with your frame as the texture. Since there is no logic in your passthrough shader code, the performance should be almost the same as BIT BLIT. – dev.bmax Sep 17 '21 at 07:31
  • Can I use https://stackoverflow.com/a/65932521 PixelCopy and get Surface of ImageReader, passing that to mediacodec encoder? Will it be efficient? – zeitgeist Feb 25 '22 at 19:40
  • 1
    Some notes: 1. you will need to set the encoder up for the correct color format and feed pictures manually. 2. this requires making 2 copies of the picture (once from Surface to Bitmap and again from Bitmap to encoder). – dev.bmax Feb 26 '22 at 18:29
  • But won't `ram_location1 to ram_location2 and ram_location2 to ram_location3` be faster than `ram_location1 to vram_location1 and vram_location1 to ram_location2` https://stackoverflow.com/questions/47648900/memory-copy-speed-comparison-cpu-gpu? I am concerned with snapdragon 865. – zeitgeist Feb 26 '22 at 21:59
  • 1
    This is something that you have to measure. Caching is a strong factor. – dev.bmax Feb 27 '22 at 08:22
0

You can't make your surfaceView bigger than the screen. Although there are multiple ways to do this in different manner but directly you can't reuse surfaceview to encode after display to screen.

Skanda Bhardwaj
  • 1
  • 1
  • I do not want to make my SurfaceView bigger than my screen. And what do you mean by "there are multiple ways to do this in different manner"? – zeitgeist Aug 13 '21 at 11:47