3

Hello everybody and thank you for reading !

Here is my problem : I have a program piping raw video frames to the standard output. This program is using OpenCV to capture and process the video and outputs directly the processed frames. The loop is synced to the framerate I chose. I'm using ffmpeg to read from the standard input and everything works fine for the video. But now that I added the sound I have a big problem : a growing delay is occuring and I really need to get rid of it. So here is my idea, but I really need your help :

I have to find a way to include a timestamp information to the raw video. To be understandable by ffmpeg, it needs to be a known raw video compatible container. And then I will need to use the container API and pipe it to the standard output in my program. I really do not know what to use in the jungle of video formats and codecs, and I don't event know how to enable timestamp synchronizing in ffmpeg...

If anyone has an idea, i am really interested here. For information, here is the command line i use to pipe the raw video :

./myprogram | ffmpeg -y -f alsa -i pulse -ac 2  -f rawvideo -vcodec rawvideo -r 24 -s 640x480 -pix_fmt bgr24 -i - -vcodec libx264 -pix_fmt yuv420p -r 24 -f flv -ar 44100 out.flv;

Thand you very much,

Roland

rkohser
  • 139
  • 3
  • 9

2 Answers2

0

The delay between video and audio is an old problem in the multimedia space. There is no silver bullet to solve it, but you may try to use different codecs (especially newer ones, and not from microsoft). You may find lower delays, and, for decent video lengths (1 hour), nothing bothersome.

Sam
  • 19,708
  • 4
  • 59
  • 82
0

The easy way out is to process the audio & video files in segments, say cut 30mins of video and audio. Since streams are desync, you can control the it with ffmpeg, see guide here or here, the nice thing is you don't need two files (streams) as ffmpeg can work with source from the SAME file.

Once you have figured out the delay, repeat for the next segment and so on.

Sometimes the audio may be longer than 30 mins, say 33mins. Then I'd use 'Audacity' to squeeze the length back to 30mins before merging.

Alvin K.
  • 4,329
  • 20
  • 25
  • Thank you Alvin, your links are very helpful ! But what I forgot to tell about and which is pretty important is that I use ffmpeg for streaming ... In the final application I will replace the flv file name by a rtmp url, so I can't afford to do post processing with audacity for example. But in your links, people talk about the "async" option of ffmpeg which ""Stretches/squeezes" the audio stream to match the timestamps" according to the ffmpeg documentation. Do you know how to include a timestamp in my raw video input ? – rkohser Nov 14 '11 at 11:11
  • I have not dealt with real-time 'stretch/squeeze' of audio stream, hence it is out of my knowledge domain (have seen examples out there but can't remember the site offhand nor tried it myself. Keep googling for it). – Alvin K. Nov 22 '11 at 06:13