4

I am trying to solve a A/V sync problem. The Video will got 1 sec delay than Audio. (See my note below please)

From Android Media Framework part, Could I just delay the audio timestamp to let it sync with video, where should i start? Is it AudioSource? MPEG4Writer? MediaCodecSource?

Please help!!

[The Case is]: If there is a 5 sec video. The current situation:

Video part on 0-1 sec will freeze, 1-5 sec stays normal. The video frame starts from 1.00s is actually frame from the 0.00s.

Audio part stays normal in whole scene. So Video part will be 1 sec delay with Audio part.

Video part on 1.00s actually match with Audio part on 0.00s.

[Failed Try]

I try if I just move the StartTimeUs on Video Part on MPEG4Writer, the A/V will sync, but in the end on file, there will be 1.00s with Video freeze, and normal audio.

[MyThought]

If I can pause Audio 1 sec, which will move Audio 0.00s to 1.00s, it will sync. But I don't know the code i can change for this method. Please help!

My Notes

user2871337
  • 115
  • 11
  • You need to record voice and video separately. For the audio recording you need to use OpenGLEs. For the video recording without voice you can use the Camera class and Surfaceview. then mix two of them. – Amit Thaper Nov 25 '16 at 06:34

1 Answers1

0

I am starting with the assumption that you are using a MediaCodec encoder and feeding the output into an MediaMuxer to produce a video file.

It sounds like this might be related to a similar issue I faced in this question. I believe that an audio MediaCodec encoder will essentially ignore any timestamps you throw at it; so I don't think that changing the audio timestamps will solve your problem. However, it sounds like you could solve the problem by correcting the video frame timestamps to be at an earlier time (as synchronized with the audio) after the frame leaves the encoder but before it reaches the muxer.

Community
  • 1
  • 1
Grant
  • 442
  • 5
  • 19
  • 1
    I have updated the question. The situation is Video will always 1 sec delay with Audio. I tried changing the Video StartTimeUs to 1sec on MPEG4Writer. It will sync until the last 1 second of File. I got freeze video but normal audio... Help~ – user2871337 Feb 11 '17 at 11:36
  • Could you share the code " Correcting the video frame timestamps to be at an earlier time" Assume the Video is 1 second delay than Audio. – user2871337 Feb 12 '17 at 01:39
  • In my case, video frames were drawn onto a Surface which served as input for the MediaCodec video encoder. So my code used pairs of Surface.lockCanvas() (or lockHardwareCanvas() on newer devices) and Surface.unlockCanvasAndPost(). Since the unlock method doesn't allow passing a timestamp, I instead manually put in a timestamp in the BufferInfo for each call to MediaMuxer.writeSampleData(). – Grant Feb 24 '17 at 02:11
  • @Grant what was your solution? I am facing the same situation. My video feed is coming in via OpenGLS, but my audio is captured with AudioRecord. Audio plays earlier than video by about 500ms. Ripping my hair out trying to sync the two. I also don't want to assume the ~500ms delay will be the same on all devices. – JCutting8 Oct 26 '19 at 03:37