I've recently started using libVLC in an Android application with the intent of replacing a commercial SDK that we're paying a lot for, but not seeing the results that we'd hoped for. The application requires viewing RTSP streams at as-close-to real-time as possible. 500ms or better ideally (depending on the tablet), without that latency drifting over time.
The process of switching out the commercial SDK for libVLC was almost seamless and it worked right away, with the exception of the latency being a few seconds (without touching any default settings). It connects to the RTSP streams really fast, and doesn't drop connections.
I've spent a couple of days tweaking the various settings to try to reduce the latency as much as possible. In some cases, I get 300ms latency, that eventually drifts to a few seconds of latency, before the stream drops and restarts (and the latency dance begins again). In other cases (I guess when I set the network cache too low), I get a log full of errors and never get a picture.
My current settings are:
val media = Media(libVLC, Uri.parse(streamUrl))
media.setHWDecoderEnabled(true, false)
media.addOption(":network-caching=300")
media.addOption(":clock-jitter=0")
media.addOption(":clock-synchro=0")
If I set network-caching to anything less than 200, I never see an image (I read somewhere this might be because the tablet's decoder latency is higher than 200ms).
Anyways, I've found dozens of potential answers spanning a decade, some are similar, some conflicting, and some using deprecated flags. Some notable ones:
2019: https://forum.videolan.org/viewtopic.php?t=149511
m.AddOption(":network-caching=150");
m.AddOption(":clock-jitter=0");
m.AddOption(":clock-syncro=0");
2018: Android LibVLC options do not work
options.add("--network-caching=50");
options.add("--clock-jitter=0");
options.add("--clock-synchro=0");
2017: Reduce delay when playing rtp stream with libvlc on Android
LibVLC libVlc = new LibVLC(context, arrayListOf(
"--file-caching=150",
"--network-caching=150",
"--clock-jitter=0",
"--live-caching=150",
"--clock-synchro=0",
"-vvv",
"--drop-late-frames",
"--skip-frames"
));
...OR...
media.setHWDecoderEnabled(true, false);
media.addOption(":network-caching=150");
media.addOption(":clock-jitter=0");
media.addOption(":clock-synchro=0");
2015: https://forum.videolan.org/viewtopic.php?f=35&t=124932&p=420020&hilit=latency#p420020
Set network cache to 500, and disable HW acceleration
2015: Decrease delay during streaming and live streaming methods
mLibvlc.setDevHardwareDecoder(LibVLC.DEV_HW_DECODER_AUTOMATIC);
mLibvlc.setHardwareAcceleration(LibVLC.HW_ACCELERATION_DISABLED);
mLibvlc.setNetworkCaching(150);
mLibvlc.setFrameSkip(true);
2013: Stream desktop over RTP using VLC with the lowest latency possible
Related to Desktop VLC and tweaking FFMPEG
And of course, we have the immense VLC flags list here: https://wiki.videolan.org/VLC_commaand-line_help/
So, in conclusion, is there a canonical method for libVLC 3+ on Android that we should collectively use for real-time streaming applications?
In my case, zero caching and dropping late frames, in order to stay as close as possible to real-time is ideal (even with janky frames). But, I do also understand that for most people, stable streaming (with occasional, sporadic frame drops) would be preferred.
I originally posted on the VideoLAN forums, 3 days ago.
