3

I am currently using ffmpeg to convert a custom container media format to mp4. It is straightforward to dump all the h.264 frames to one file and the aac audio to another. Then I can combine the two and create an mp4 file with ffmpeg.

The problem is that the video source isn't always perfect. From time to time frames are dropped or late etc. This causes an A/V sync issue since the pts is generated using a constant rate by ffmpeg. The source format I am using has the PTS value but I cant figure out a way to pass it to ffmpeg with the raw h.264 frames.

I suppose it would be possible to create a demuxer for the custom format, but it seems like a lot effort. I looked into ffmpeg's .nut container format thinking that I might be able to convert from the custom container to .nut first. Unfortunately it seems more complex than it looks on the surface.

It seems like there should be an easy way to pass a frame and its PTS value to ffmpeg, but I haven't come across it yet. Any help would be appreciated.

Here is the ffmpeg command I am using

ffmpeg -f s16le -ac 1 -ar 48k -i source.audio -framerate 20 -i source.video -c:a aac -b:a 64k -r 20 -c:v h264_nvenc -rc:v vbr_hq -cq:v 19 -n out.mp4
thecaptain0220
  • 2,098
  • 5
  • 30
  • 51
  • Can you show a code sample for how you're sending the frames to FFmpeg? It seems very much that you need to create demuxer for your custom format. – Brad May 19 '20 at 17:01
  • I added the command. I am manually specifying the input framerate. I think you are right that the demuxer is what I need to do. I was trying to see if there were any shortcuts that didnt involve building ffmpeg and figure out everything needed to write the demuxer. – thecaptain0220 May 19 '20 at 17:32
  • Ah, you're using the CLI now? Yeah, getting FFmpeg built is hard enough as it is. Hopefully someone else has a better answer for you. – Brad May 19 '20 at 17:36
  • See https://stackoverflow.com/q/28086775 – Gyan May 19 '20 at 19:23

0 Answers0