How to extract fully synced audio with ffmpeg?

Question

I have an MP4 file with video and audio. I need to extract the audio stream only to an MP3 file.

However it is critical for our application that the timing / synchronization of the audio file is exactly aligned with the original video, as we cross-correlate audio features based on timing.

I currently use this command:

ffmpeg -y 
   -v error 
   -map 0:a ./the-output-file-mp3
   -map 0:v ~/a-dummy-video-output-file.mp4
   -i ~/the-input-file.mp4
   -acodec copy

as I read somewhere that mapping both 0:a and 0:v would guarantee alignment & synchronization. I then discard the video file as it is not needed.

However it is orders of magnitude faster to just extract the audio instead:

ffmpeg -y 
   -v error 
   -map 0:a ./the-output-file-mp3
   -i ~/the-input-file.mp4
   -acodec copy

But I don't know if this guarantees alignment, because I read on this page that:

If you only extract audio from a video stream, the length of the audio may be shorter than the length of the video. To make sure this doesn't happen, extract both audio AND video with the same call to ffmpeg, e.g. "ffmpeg -i vid.avi -map 0:a audio.wav -map 0:v onlyvideo.avi

However perhaps I am putting too much weight on that comment. I don't care if the audio clip finishes early, but it is critical that it does not start late, or get de-synced from the original video at any point.

What is the most efficient way with ffmpeg to extract the audio stream with a guarantee that it is 100% aligned in time to the video (and starts at exactly the same time)?

Will this method guarantee alignment?

ffmpeg -y 
   -v error 
   -i ~/the-input-file.mp4
   -vn
   -acodec copy 
   ~/the-output-file.mp4

My speculative answer is your last snippet should be fine. If your input streams could have different start times, then add `-copyts` input option. Compared to lossless formats (wav, flac, etc.) I've seen slight misalignment in mp3. So, if your streams are always starting at 0, using a lossless format should work as well (if allowed). — kesh, Dec 01 '22 at 14:18

score 0 · Accepted Answer · answered Jan 02 '23 at 11:51

From my validation over 100 video files, the following one gave correct alignment and is lightening fast:

ffmpeg -y 
   -v error 
   -i ~/the-input-file.mp4
   -vn
   -acodec copy 
   ~/the-output-file.mp4

However my video files have strong alignment of streams already and -copyts may be necessary for messier files.

How to extract fully synced audio with ffmpeg?

1 Answers1