0

Using OpenCV 4.5.2 + FFMPEG on an android app

I'm trying to convert an .avi video file into a .mp4 file using x264, by running

ffmpeg -i input.avi -c:v libx264 output.mp4

The transcoding is processed correctly but, when I play the video, the colors are a bit... saturated?

This transcoding is part of the following flow:

  1. Grab a .mov video file
  2. Use OpenCV VideoCapture and VideoWriter to write text on the video frames (output is .avi)
  3. Then I need to convert .avi file into .mp4 so it's reproducible on exoplayer.

In step 2. I'm looping all video frames and writing them to a new file, writing a text on them.

val videoWriter = VideoWriter(
            outputFilePath,
            VideoWriter.fourcc('M', 'J', 'P', 'G'),
            15.0,
            Size(1920.0, 1088.0),
            true
        )

 val frame = Mat()
 videoCapture.read(frame)
 Imgproc.putText(
    frame,
    "This is a text",
    Point(200.0, 200.0),
    3,
    5.0,
    Scalar(
        255.0,
        124.0,
        124.0,
        255.0
    ), 
    1
)
videoWriter.write(frame)

I know that step 2. is probably not corrupting the frames because in my sample app, I'm displaying all frames in an ImageView, and they all match the original .mov video. So, my guess is that the issue is occurring on 3.

I'm using 'com.arthenica:mobile-ffmpeg-min-gpl:4.4' for android to execute the FFMPEG command as follows:

FFmpeg.executeAsync("-i $outputFilePath -c:v libx264 -y ${mp4File.path}")

where outputFilePath is the path for the .avi file and mp4File is an existing empty .mp4 file.

So I guess what I'm looking for is a way to have a lossless video color transcoding between .avi and .mp4 files.

Here's a screenshot of my sample app. The image on top is the last frame of the .avi video. The image on the bottom is the last frame played on a video player for the .mp4 transcoded video. This frame color difference is noticeable throughout the whole video.

EDIT: After some digging, I found out that the issue is that the VideoWritter is messing with the RGB colors. I still don't know the reason why this is happeninng.

enter image description here

Tiago Ornelas
  • 1,109
  • 8
  • 21
  • Looks like the red and blue color planes got switched. Show the complete log from the ffmpeg command, and if possible provide a link to a short example avi input that has this behavior. – llogan Apr 16 '21 at 16:23
  • I can't really access the .avi file since it's converted from the .mov file on the app's cache directory. But here are the logs https://gist.github.com/TOrnelas/b6176d265c9779af9dac0cf30e09f2a8. I have also tried with other a mp4 video as origin (i.e. mp4 -> avi -> mp4) and the same happens. See the image output for that here https://photos.app.goo.gl/AbV4Duqr1oYy6utQ7 – Tiago Ornelas Apr 16 '21 at 16:40
  • Try `ffmpeg -i input.avi -c:v libx264 -vf format=yuv420p output.mp4` – llogan Apr 16 '21 at 16:42
  • Tried now. Same result – Tiago Ornelas Apr 16 '21 at 16:46
  • Ok. I was finally able to access the .avi file which is already messed up :D So, the OpenCV VideoWritter is messing this up! You can see the avi video and final mp4 video here https://photos.app.goo.gl/T2G8Q81axRxjq9dm6 – Tiago Ornelas Apr 16 '21 at 17:40
  • If you want to [add text just use ffmpeg](https://stackoverflow.com/questions/17623676/text-on-video-ffmpeg/17624103#17624103) and you can avoid opencv and the extra step. – llogan Apr 16 '21 at 17:49
  • I thought about that. However, in the final app version, the text changes depending on the frame kndex. There's now way to to that with ffmpeg directly I'm guessing – Tiago Ornelas Apr 16 '21 at 19:22
  • @llogan I figured it out. Check my answer. Thanks for helping me debug this issue – Tiago Ornelas Apr 16 '21 at 22:36
  • drawtext in ffmpeg would not be practical for arbitrary per frame text. – llogan Apr 18 '21 at 04:23

1 Answers1

1

Figured it out myself with some debug assitance from @llogan.

So, it looks like VideoCapture exports frames with BGR format, thus the Red and Blue colors being switched out. In order to fix my issue all I had to do was to convert the frame from BGR to RGB using the OpenCV utility method:

 val frame = Mat()
 val frame1 = Mat()
 videoCapture.read(frame)

 Imgproc.cvtColor(frame, frame1, Imgproc.COLOR_BGR2RGB)
 videoWriter.write(frame1)
Tiago Ornelas
  • 1,109
  • 8
  • 21