0

How can I capture the screen as raw rgba buffers ( without encoding ) using MediaProjectionManager ?

I have seen many examples of how to capture the screen and encode it using MediaCodec but I want to use my own encoder instead.

jacob
  • 1,397
  • 1
  • 26
  • 53
  • 2
    One approach is to render the frames with GLES and extract them with `glReadPixels()`, like http://bigflake.com/mediacodec/#ExtractMpegFramesTest, though that feels like a bit of a long walk. In theory you can create an ImageReader and access the data through that, but I haven't tried it -- http://stackoverflow.com/questions/31687259/ seemed to get it working. – fadden Aug 07 '15 at 17:28
  • Would ImageReader be able to handle 20 fps frame rate ? – jacob Aug 09 '15 at 13:16
  • ImageReader just wraps the Surface, it doesn't copy data, so it doesn't add much in the way of overhead. On the other hand, it sounds like you will be processing video frames with your application in software (as opposed to a hardware-based video encoder), which can be expensive. So whether or not your app can handle 20 fps depends on what your app does with the frames. – fadden Aug 09 '15 at 14:44

2 Answers2

1

I ended up using ImageReader as fadden suggested

jacob
  • 1,397
  • 1
  • 26
  • 53
0

MediaCodec supports RAW MIME type, MIMETYPE_VIDEO_RAW, if you look at http://developer.android.com/reference/android/media/MediaFormat.html#MIMETYPE_VIDEO_RAW

However, if you even want to use your custom codec that can replace MediaCodec, the whole thing should be hooking up to the mInputSurface. 

mMediaProjection.createVirtualDisplay("Recording Display", 
            screenWidth, screenHeight, screenDensity, 0 /* flags */, 
            mInputSurface, null /* callback */, null /* handler */);

You may notice that Surface API has a lockCanvas, it probably is the API for writing to your codec via Canvas.

Surface: https://developer.android.com/reference/android/view/Surface.html

Canvas: https://developer.android.com/reference/android/graphics/Canvas.html

I will be interested to see your result.

phdfong - Kenneth Fong
  • 870
  • 1
  • 9
  • 10