Android ExoPlayer 2 - How to improve fps when rendering many videos?

Question

I'm using ExoPlayer 2 to create an app on Android that renders and plays multiple video simultaneously. However, when the number of videos displayed exceeds a certain limit (3-4 videos on my Xiaomi Note 11 Pro), the videos start to drop fps. I tried to limit the players' max bitrate but nothing changed.

Player player = new ExoPlayer.Builder(context).build();
TrackSelectionParameters parameters = player.getTrackSelectionParameters();
TrackSelectionParameters newParameters = parameters.buildUpon().setMaxVideoBitrate(2000000).build(); // 2 Mbps
player.setTrackSelectionParameters(newParameters);

How can I improve fps when app render many videos?

Do all the videos start to drop frames or are one or more still ok? — Mick, Jun 20 '23 at 09:07
@Mick all of them. Actually, the app itself started lagging too, not just the videos. — Hoài Vũ Lê, Jun 20 '23 at 10:12

score 1 · Answer 1 · answered Jun 20 '23 at 13:01

Ultimately, I think you are simply running out of memory/processing power and the system is trying to degrade gracefully by dropping frames.

When you have multiple videos also, you are more likely to go beyond the limits of any HW codecs and have to fall back onto SW codecs.

Many video architectures are designed to try to degrade gracefully by dropping frames if they cannot display them in time.

I'm not sure what the latest status with Android is but there is some excellent information here from 5 or 6 years ago: https://stackoverflow.com/a/26345345/334402

See in particular the note about the move towards dropping frames which would not be displayed at the time intended:

The solution Android is moving toward for A/V sync is to have the app tell SurfaceFlinger when it wants the frame to be displayed. If SurfaceFlinger misses the deadline, it drops the frame.

The root issue, needing to display more videos than a device can handle comfortably, is quite common and not restricted to Android.

One possible workaround, which may not be applicable for your case, is to combine the videos into a single stream on the server side and then send that single stream to the device. You can still allow a user to expand or 'move to stage' a particular video by using a touch grid over the combined video and adding a second stream for the selected video if a user selects it.

Android ExoPlayer 2 - How to improve fps when rendering many videos?

1 Answers1