How can I display multiple images from a video without tanking performance?

Question

I'm trying to create a control that will allow a user to create clips of a video. The modeling for a clip contains a Uri to the video and a start/end TimeSpan:

public class ClipModel
{
    public Uri Source { get; }
    public TimeSpan Start{ get; }
    public TimeSpan End { get; }
}

I want to display a preview of all clips in a control, but my implementation has performance issues. I believe this is because I create a MediaElement for each clip, which forces the file to be loaded many times. I expect to have dozens of clips for a given video, so loading a video file ~20 times really tanks performance.

The xaml for all that is:

<ScrollViewer Grid.Row="3" Padding="0,20,0,0" Background="White" FocusVisualStyle="{x:Null}"
              HorizontalScrollBarVisibility="Visible">
    <ItemsControl x:Name="Clips">
        <ItemsControl.ItemsPanel>
            <ItemsPanelTemplate>
                <StackPanel Orientation="Horizontal"/>
            </ItemsPanelTemplate>
        </ItemsControl.ItemsPanel>
        <ItemsControl.ItemTemplate>
            <DataTemplate DataType="{x:Type models:ClipModel}">
                <MediaElement Source="{Binding  VideoPath}"
                          helpers:MediaElementHelper.Position="{Binding Start}"
                          ScrubbingEnabled="True"
                          LoadedBehavior="Pause"
                          Height="50"
                          Width="100"
                          Margin="5"/>
            </DataTemplate>
        </ItemsControl.ItemTemplate>
    </ItemsControl>
</ScrollViewer>

Is there a better control for viewing a single frame from a video which won't tank performance when there are many of them?

The model allows for many clips to come from different video files. However, there will generally be many clips for a single video, rather than from many different videos.

Given that, perhaps a better question would be: is there some way I can load a video into memory once, and use that single Stream/byte[] across many MediaElements to get all the thumbnails?

score 0 · Answer 1 · answered Nov 07 '22 at 03:19

I just thought of a workaround. It should work for my specific use-case, but I doubt it would be satisfying to most people who have a similar issue.

The solution would be to use MediaElement.GetScreenshot when a clip is created, save that image to disk, and write down the path on the model. This will allow me to show a preview image for all the clips without having to load a single video dozens of times.

The obvious drawbacks are that this takes up space (~1 mb per clip) and managing the files needs to be handled intelligently to prevent scalability issues. However, given what seems to be the state of things from this question, I don't expect to find anything that will work as cleanly as this.

How can I display multiple images from a video without tanking performance?

1 Answers1