Rotate the matrix (frames) of a video in a GlSurfaceView.Renderer implementation

Question

I am using GPUVideo-android library to apply filters to my ExoPlayer backed video. I use filters like Brightness, Contrast, Pixelation etc.

See footnotes for context or background information.

When I use GPUPlayerView from that library, the played/rendered video is not rotated according to video's metadata.

This is because GPUPlayerView uses a GPUPlayerRenderer to render the frames onto a SurfaceView. So, this renderer has no way to check the orientation of video.

If I created a method like renderer.setRotation(Int), what will be the body of this method? I have no prior experience with OpenGl/EGL stuff. I want to learn more about it (to continue future development), but I don't have time at present.

I have tried to apply solution from this answer which tells me to add:

Matrix.rotateM(mTmpMatrix, 0, 90, 0, 0, 1);
Matrix.translateM(mTmpMatrix, 0, 0, -1, 0);

I added above code before line number 154, where I replaced mTmpMatrix with MVPMatrix in onDrawFrame(). But, I wasn't successful. I tried to write the snippet below line number 156 too.

I have, also, tried to create a tempMatrix of float[16]. And then, do this in onDrawFrame():

GLES20.glClear(GL_COLOR_BUFFER_BIT);

tempMatrix = MVMMatrix.clone();

Matrix.rotateM(tempMatrix, 0, 90, 0, 0, 1);
Matrix.translateM(tempMatrix, 0, 0, -1, 0);

Matrix.multiplyMM(tempMatrix, 0, VMatrix, 0, MMatrix, 0);
Matrix.multiplyMM(tempMatrix, 0, ProjMatrix, 0, MVPMatrix, 0);

previewFilter.draw(texName, tempMatrix, STMatrix, aspectRatio);

But, I wasn't successful.

So, how can I write a method like renderer.setRotation(Int)?

EDIT 1:

I have, also, found some relevant code in iOS version of similar library, but it doesn't help me.

EDIT 2:

If I add Matrix.rotateM(MMatrix, 0, 90, 0, 0, 1); at the end of line number 134, rendering frame is rotated. But, it becomes square. So, if video was 720x1280, it becomes 720x720 leaving black space above and below the frames after I add this line.

Footnotes / Context:

When I record a video from my device, it is saved with a rotation orientation of 90 degrees. I verified this by using MediaMetadataRetriever

val extractedRotation = retriever.extractMetadata(
    MediaMetadataRetriever.METADATA_KEY_VIDEO_ROTATION
).toIntOrNull()

If the rotation is 90 or 270, I swap the width and height so that my layout shows correct aspect ratio of video.

val (correctedWidth, correctedHeight) = when (extractedRotation) {
    90, 270 -> arrayOf(extractedHeight, extractedWidth)
    else -> arrayOf(extractedWidth, extractedHeight)
}

If the video resolution is 1280x720 with a rotation of 90, corrected resolution becomes 720x1280.

When I play this video in ExoPlayer's PlayerView, video is played correctly. I think ExoPlayer respects video's orientation metadata.

Do you get answer of your question then please help me. I have same issue of rotation. — Kinal Mer, Jun 15 '20 at 05:28
I didn't get an answer to my question. Although if I learned more about graphics, I could've done this. But, I never had enough time. I was, also, using ffmpeg lib in my app, so I resorted back to properly rotating video with ffmpeg before submitting it to exoplayer. When a video with rotation-flags is passed to ffmpeg, it rotates the video correctly and clears rotation-flags on output. Takes 10~12sec for a video duration of 10sec. — rupinderjeet, Jun 15 '20 at 05:49
@KinalMer If you do find a solution, please add it as an answer here. — rupinderjeet, Jun 15 '20 at 05:50

Rotate the matrix (frames) of a video in a GlSurfaceView.Renderer implementation

EDIT 1:

EDIT 2:

0 Answers0