How much lag is expected when an android device's camera is streamed live? I checked Miracast and it has a lag of around 150-300 ms(usually its around 160-190 ms). I have a bluetooth and a wifi direct app and both lag by around 400-550 ms. I was wondering if it would be possible to reproduce or come closer to Miracast's performance. I am encoding the camera frames in H.264 and using my own custom protocol to transmit the encoded frames over a TCP connection(in case of WiFi Direct).
Asked
Active
Viewed 1,173 times
1 Answers
0
Video encoding is compute intensive so any help you can get from the hardware usually speeds things up. It is worth making sure you are using codecs which leverage the hardware - i.e avoid having your H.264 encoding all in software.
There is a video here which discusses how to access HW accelerated video codecs, although it is a little old, but will apply to older devices:
I believe that MediaCodec now provides a Java API to the Hardware Codecs (have not tried or seen speed comparison tests myself), which makes things easier:
See the note here also about using the camera's Surface preview as a speed aid:
As an aside, 500ms does not seem that bad, and you are probably into diminishing returns so the effort to reduce may be high if you can live with your current lag.
Also, I assume you are measuring the lag (or latency) on the server side? If so you need to look at how the server is decoding and presenting also, especially if you are comparing your own player with a third party one.
It is worth looking at your jitter buffer on the receiving side, irrespective of the stream etc latency - in simple terms waiting for a larger number of packets before you start playback may cause more startup delay but it also may provide a better overall user experience. This is because the large buffer will be more tolerant of delayed packets, before going into a 'buffering' mode that users tend not to like.
Its a balance, and your needs may dictate a bias one way or the other. If you were planning a video chat like application for example, then the delay is very important as it starts to become annoying to users above 200-300 ms. If you are providing a feed from a sports event, then delay may be less important and avoiding buffering pauses may give better user perceived quality.
-
hey thanks for the answer..the problem is basically with the android camera preview callback which only gets called every 50 ms or so..so i am losing a lot of time just waiting for the preview callback..i tried setting the camera fps range but that didn't work as well so i am going to try passing the surface directly now..the encoder and decoder are pretty fast but they do seem to buffer a few NAL units..what i was looking was some kind of benchmarks or something for android streaming apps...almost all streaming apps in the play store also lag which left me wondering if the lag is inevitable – thunderbird Mar 24 '15 at 17:07
-
@thunderbird how far did you get with this? – kar Apr 15 '19 at 09:42
-
@kar if i remeber correctly, i was stuck at around 200-400ms delay. But that was almost 4 years ago on a nexus 7 to Windows PC. – thunderbird Apr 16 '19 at 11:11