3

I am working on an app that captures 4K Video only using the camera2 API - No preview surface displayed and no audio recorded.

I am using an LG G5 running Android 7.0.0. The device reports FULL implementation of the camera2 API when queried. The native camera app records in 4K just fine. The reported resolutions by mediarecorder confirm the availability of 4K:

resolutions from log

My code to initialize MediaRecorder is below. I do this over using setProfile because I cannot set a profile without making the program throw exceptions because I haven't set any audio recording settings (as I don't want or care to record audio):

        vidRecorder = new MediaRecorder();
        vidRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
        vidRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
        vidRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264);
        vidRecorder.setVideoFrameRate(30);
        vidRecorder.setCaptureRate(30);

        vidRecorder.setVideoEncodingBitRate(35000000);
        vidRecorder.setVideoSize(3840, 2160);

        vidRecorder.setOutputFile(videoFilename);

        vidRecorder.prepare();

There are no issues shown in the log with the above. The output format and encoder match what VLC reports for 4K video files produced by the phone native app.

Everything above runs just fine except the produced video file is always 1920x1080p! I'm out of ideas as to why this is happening. Thanks in advance for any help.

Andy
  • 363
  • 1
  • 3
  • 6
  • Additional Info: I attempted to recording using a profile with manually set resolution: `CamcorderProfile profile = CamcorderProfile.get(0, CamcorderProfile.QUALITY_HIGH); profile.videoFrameWidth = 3840; profile.videoFrameHeight = 2160; profile.videoBitRate = (int)(profile.videoBitRate*2.8); vidRecorder.setProfile(profile);` The above code again compiles and runs without error yet the video file is still 1080p. ugh! – Andy Feb 12 '18 at 16:09
  • Info: It seems every resolution listed in the mediarecorder output list works except for 3840x2160p and above (3264x2448 works fine and produces a matching res file). Setting any resolution above that leads to the program executing fine but producing a 1080p video file. How frustrating. – Andy Feb 12 '18 at 16:56
  • whelp, as far as I can gather the reason this isn't working is that the manufacturer (LG) has locked out 4K recording to everything but the native app. All other indicators I mentioned that show 4K recording is possible are moot - the phone simply won't record 4K no matter what unless I use the native app. Great. – Andy Feb 12 '18 at 22:16

1 Answers1

0

You have to enable OpenGL ES 3.0 or greater. And then override the SurfaceView as GLSurfaceView for the Camera2 API preview with a custom class using GLES30 instead of GLES20 or GLES10.

Ideally, you would use GLES32 because it supports 8K.

https://android.googlesource.com/platform/development/+/abededd/samples/browseable/Camera2Video/src/com.example.android.camera2video/Camera2VideoFragment.java

This Camera2Video example uses AutoFitTextureView:

https://github.com/googlearchive/android-Camera2Basic/blob/master/Application/src/main/java/com/example/android/camera2basic/AutoFitTextureView.java

You would want to use GLSurfaceView:

https://android.googlesource.com/platform/frameworks/base/+/master/opengl/java/android/opengl/GLSurfaceView.java

https://android.googlesource.com/platform/frameworks/base/+/master/graphics/java/android/graphics/SurfaceTexture.java

https://android.googlesource.com/platform/frameworks/base/+/android-4.3_r2.1/core/java/android/view/Surface.java

It’s tedious but not difficult. You just replace GLES10 or GLES20 with GLES30 or ideally GLES32.

EDIT:

I think the Android OS automatically uses the maximum OpenGL ES version which is why MediaPlayer can play 4K video on OpenGLES 3.0 but it shows a black image for OpenGL ES 2.0.

If your camera app isn’t capturing 4K, trying playing a 4K video in MediaPlayer and see what happens:

Android: Play local video with mediaplayer

Otherwise, you would have to find the usage of updateTexImage() from SurfaceTexture in the Camera2 APIs and use GLES30. It’s not really clear where OpenGL ES is being used, but it’s clearly referenced in SurfaceTexture.java

The Camera2 ImageCapture seems to bypass Surface altogether. Although I could be mistaken.

And to the point about MediaPlayer not being able to play 4K video, LG G5 might only support OpenGL ES 2.0.

4K requirements

  1. Device must support OpenGL ES 3.0 or greater
  2. Android OS must invoke OpenGL ES 3.0 or greater on Surface, Surface Texture (try API 31)
  3. Camera2 private APIs must invoke OpenGL ES 3.0 or greater (try API 31)

Edit

This answer talks about modifying a SurfaceView using OpenGL.

Modifying camera output using SurfaceTexture and OpenGL

Michael N
  • 436
  • 5
  • 6