Loading a large amount of images to be displayed in a WrapPanel

Question

I am using Entity Framework Code First

I have a Movie like so:

public class Movie
{
        public byte[] Thumbnail { get; set; }
        public int MovieId { get; set; }
}

And a Collection of Movies like so:

public class NorthwindContext : DbContext
{
    public DbSet<Movie> Movies { get; set; }
}

I have a MovieViewModel like so:

public class MovieViewModel
{
        private readonly Movie _movie;

        public MovieViewModel(Movie movie)
        {
            _movieModel = movieModel;
        }

        public byte[] Thumbnail { get { return _movie.Thumbnail; } }
}

When my App starts:

public ObservableCollection<MovieViewModel> MovieVms = 
                      new ObservableCollection<MovieViewModel>();

foreach (var movie in MyDbContext.Movies)
     MovieVms.Add(new MovieViewModel(movie));

I have 4000 movies. This process takes 25 seconds. Is there a better/faster way to do this?

My main page uses the thumbnails like so, but to be clear this loading time happens before anything UI related:

MyView = new ListCollectionView(MovieVms);

<ListBox ItemsSource="{Binding MyView}" />

Also my memory usage goes through the roof. How should I be loading these images? I need a full collection of view models off the bat to enable sorting, filtering, searching, but I only need the thumbnails of the items visible in my wrap panel.

EDIT---

Thanks Dave for a great answer. Can you elaborate on "make it an association (aka navigation property)"

var thumbnail = new Thumbnail();
thumbnail.Data = movie.GetThumbnail();
Globals.DbContext.Thumbnails.Add(thumbnail);
Globals.DbContext.SaveChanges();
movie.ThumbnailId = thumbnail.ThumbnailId;
Globals.DbContext.SaveChanges();

I can run that code with no errors, but my property in my MovieViewModel

public new byte[] Thumbnail { get { return _movie.Thumbnail.Data; } }

always has a null Thumbnail and errors as soon as my UI accesses it. A breakpoint on movie.ThumbnailId is never hit. Do I have to load the association manually?

Answer 1

I think you are essentially asking how to do several different things:

• Load the entire list of movies quickly, to allow for sorting and filtering in the UI
• Display the movie thumbnails in the UI but only when they are scrolled into view
• Keep memory usage to a minimum
• Display the UI as quickly as possible after the application starts

Load the movies quickly

First off, as @Dave M's answer states, you need to split the thumbnail into a separate entity so that you can ask Entity Framework to load the list of movies without also loading the thumbnails.

public class Movie
{
    public int Id { get; set; }
    public int ThumbnailId { get; set; }
    public virtual Thumbnail Thumbnail { get; set; }  // This property must be declared virtual
    public string Name { get; set; }

    // other properties
}

public class Thumbnail
{
    public int Id { get; set; }
    public byte[] Image { get; set; }
}

public class MoviesContext : DbContext
{
    public MoviesContext(string connectionString)
        : base(connectionString)
    {
    }

    public DbSet<Movie> Movies { get; set; }
    public DbSet<Thumbnail> Thumbnails { get; set; }
}

So, to load all of the movies:

public List<Movie> LoadMovies()
{
    // Need to get '_connectionString' from somewhere: probably best to pass it into the class constructor and store in a field member
    using (var db = new MoviesContext(_connectionString))
    {
        return db.Movies.AsNoTracking().ToList();
    }
}

At this point you will have a list of Movie entities where the ThumbnailId property is populated but the Thumbnail property will be null as you have not asked EF to load the related Thumbnail entities. Also, should you try to access the Thumbnail property later you will get an exception as the MoviesContext is no longer in scope.

Once you have a list of Movie entities, you need to convert them into ViewModels. I'm assuming here that your ViewModels are effectively read-only.

public sealed class MovieViewModel
{
    public MovieViewModel(Movie movie)
    {
        _thumbnailId = movie.ThumbnailId;
        Id = movie.Id;
        Name = movie.Name;
        // copy other property values across
    }

    readonly int _thumbnailId;

    public int Id { get; private set; }
    public string Name { get; private set; }
    // other movie properties, all with public getters and private setters

    public byte[] Thumbnail { get; private set; }  // Will flesh this out later!
}

Note that we're just storing the thumbnail ID here, and not populating the Thumbnail yet. I'll come to that in a bit.

Load thumbnails separately, and cache them

So, you've loaded the movies, but at the moment you haven't loaded any thumbnails. What you need is a method that will load a single Thumbnail entity from the database given its ID. I would suggest combining this with a cache of some sort, so that once you've loaded a thumbnail image you keep it in memory for a while.

public sealed class ThumbnailCache
{
    public ThumbnailCache(string connectionString)
    {
        _connectionString = connectionString;
    }

    readonly string _connectionString;
    readonly Dictionary<int, Thumbnail> _cache = new Dictionary<int, Thumbnail>();

    public Thumbnail GetThumbnail(int id)
    {
        Thumbnail thumbnail;

        if (!_cache.TryGetValue(id, out thumbnail))
        {
            // Not in the cache, so load entity from database
            using (var db = new MoviesContext(_connectionString))
            {
                thumbnail = db.Thumbnails.AsNoTracking().Find(id);
            }

            _cache.Add(id, thumbnail);
        }

        return thumbnail;
    }
}

This is obviously a very basic cache: the retrieval is blocking, there is no error handling, and the thumbnails should really be removed from the cache if they haven't been retrieved for a while in order to keep memory usage down.

Going back to the ViewModel, you need to modify the constructor to take a reference to a cache instance, and also modify the Thumbnail property getter to retrieve the thumbnail from the cache:

public sealed class MovieViewModel
{
    public MovieViewModel(Movie movie, ThumbnailCache thumbnailCache)
    {
        _thumbnailId = movie.ThumbnailId;
        _thumbnailCache = thumbnailCache;
        Id = movie.Id;
        Name = movie.Name;
        // copy other property values across
    }

    readonly int _thumbnailId;
    readonly ThumbnailCache _thumbnailCache;

    public int Id { get; private set; }
    public string Name { get; private set; }
    // other movie properties, all with public getters and private setters

    public BitmapSource Thumbnail
    {
        get
        {
            if (_thumbnail == null)
            {
                byte[] image = _thumbnailCache.GetThumbnail(_thumbnailId).Image;

                // Convert to BitmapImage for binding purposes
                var bitmapImage = new BitmapImage();
                bitmapImage.BeginInit();
                bitmapImage.StreamSource = new MemoryStream(image);
                bitmapImage.CreateOptions = BitmapCreateOptions.None;
                bitmapImage.CacheOption = BitmapCacheOption.Default;
                bitmapImage.EndInit();

                _thumbnail = bitmapImage;
            }

            return _thumbnail;
        }
    }
    BitmapSource _thumbnail;
}

Now the thumnail images will only be loaded when the Thumbnail property is accessed: if the image was already in the cache, it will be returned immediately, otherwise it will be loaded from the database first and then stored in the cache for future use.

Binding performance

The way that you bind your collection of MovieViewModels to the control in your view will have an impact on perceived loading time as well. What you want to do whenever possible is to delay the binding until your collection has been populated. This will be quicker than binding to an empty collection and then adding items to the collection one at a time. You may already know this but I thought I'd mention it just in case.

This MSDN page (Optimizing Performance: Data Binding) has some useful tips.

This awesome series of blog posts by Ian Griffiths (Too Much, Too Fast with WPF and Async) shows how various binding strategies can affect the load times of a bound list.

Only loading thumbnails when in view

Now for the most difficult bit! We've stopped the thumbnails from loading when the application starts, but we do need to load them at some point. The best time to load them is when they are visible in the UI. So the question becomes: how do I detect when the thumbnail is visible in the UI? This largely depends on the controls you are using in your view (the UI).

I'll assume that you are binding your collection of MovieViewModels to an ItemsControl of some type, such as a ListBox or ListView. Furthermore, I'll assume that you have some kind of DataTemplate configured (either as part of the ListBox/ListView markup, or in a ResourceDictionary somewhere) that is mapped to the MovieViewModel type. A very simple version of that DataTemplate might look like this:

<DataTemplate DataType="{x:Type ...}">
    <StackPanel>
        <Image Source="{Binding Thumbnail}" Stretch="Fill" Width="100" Height="100" />
        <TextBlock Text="{Binding Name}" />
    </StackPanel>
</DataTemplate>

If you are using a ListBox, even if you change the panel it uses to something like a WrapPanel, the ListBox's ControlTemplate contains a ScrollViewer, which provides the scroll bars and handles any scrolling. In this case then, we can say that a thumbnail is visible when it appears within the ScrollViewer's viewport. Therefore, we need a custom ScrollViewer element that, when scrolled, determines which of its "children" are visible in the viewport, and flags them accordingly. The best way of flagging them is to use an attached Boolean property: in this way, we can modify the DataTemplate to trigger on the attached property value changing and load the thumbnail at that point.

The following ScrollViewer descendant (sorry for the terrible name!) will do just that (note that this could probably be done with an attached behaviour instead of having to subclass, but this answer is long enough as it is).

public sealed class MyScrollViewer : ScrollViewer
{
    public static readonly DependencyProperty IsInViewportProperty =
        DependencyProperty.RegisterAttached("IsInViewport", typeof(bool), typeof(MyScrollViewer));

    public static bool GetIsInViewport(UIElement element)
    {
        return (bool) element.GetValue(IsInViewportProperty);
    }

    public static void SetIsInViewport(UIElement element, bool value)
    {
        element.SetValue(IsInViewportProperty, value);
    }

    protected override void OnScrollChanged(ScrollChangedEventArgs e)
    {
        base.OnScrollChanged(e);

        var panel = Content as Panel;
        if (panel == null)
        {
            return;
        }

        Rect viewport = new Rect(new Point(0, 0), RenderSize);

        foreach (UIElement child in panel.Children)
        {
            if (!child.IsVisible)
            {
                SetIsInViewport(child, false);
                continue;
            }

            GeneralTransform transform = child.TransformToAncestor(this);
            Rect childBounds = transform.TransformBounds(new Rect(new Point(0, 0), child.RenderSize));
            SetIsInViewport(child, viewport.IntersectsWith(childBounds));
        }
    }
}

Basically this ScrollViewer assumes that it's Content is a panel, and sets the attached IsInViewport property to true for those children of the panel that lie within the viewport, ie. are visible to the user. All that remains now is to modify the XAML for the view to include this custom ScrollViewer as part of the ListBox's template:

<Window x:Class="..."
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:my="clr-namespace:...">

    <Window.Resources>
        <DataTemplate DataType="{x:Type my:MovieViewModel}">
            <StackPanel>
                <Image x:Name="Thumbnail" Stretch="Fill" Width="100" Height="100" />
                <TextBlock Text="{Binding Name}" />
            </StackPanel>
            <DataTemplate.Triggers>
                <DataTrigger Binding="{Binding Path=(my:MyScrollViewer.IsInViewport), RelativeSource={RelativeSource AncestorType={x:Type ListBoxItem}}}"
                             Value="True">
                    <Setter TargetName="Thumbnail" Property="Source" Value="{Binding Thumbnail}" />
                </DataTrigger>
            </DataTemplate.Triggers>
        </DataTemplate>
    </Window.Resources>

    <ListBox ItemsSource="{Binding Movies}">
        <ListBox.Template>
            <ControlTemplate>
                <my:MyScrollViewer HorizontalScrollBarVisibility="Disabled" VerticalScrollBarVisibility="Auto">
                    <WrapPanel IsItemsHost="True" />
                </my:MyScrollViewer>
            </ControlTemplate>
        </ListBox.Template>
    </ListBox>

</Window>

Here we have a Window containing a single ListBox. We've changed the ControlTemplate of the ListBox to include the custom ScrollViewer, and inside that is the WrapPanel that will layout the items. In the window's resources we have the DataTemplate that will be used to display each MovieViewModel. This is similar to the DataTemplate introduced earlier, but note that we are no longer binding the Image's Source property in the body of the template: instead, we use a trigger based on the IsInViewport property, and set the binding when the item becomes 'visible'. This binding will cause the MovieViewModel class's Thumbnail property getter to be called, which will load the thumbnail image either from the cache or the database. Note that the binding is to the property on the parent ListBoxItem, into which the markup for the DataTemplate is injected.

The only problem with this approach is that, as the thumbnail loading is done on the UI thread, scrolling will be affected. The easiest way to fix this would be to modify the MovieViewModel Thumbnail property getter to return a "dummy" thumbnail, schedule the call to the cache on a separate thread, then get that thread to set the Thumbnail property accordingly and raise a PropertyChanged event, thus ensuring the binding mechanism picks-up the change. There are other solutions but they would raise the complexity significantly: consider what is presented here as just a possible starting point.

Answer 2

Whenever you request an Entity from EF, it automatically loads all scalar properties (only associations are lazy loaded). Move the Thumbnail data to it's own Entity, make it an association (aka navigation property) and take advantage of the Lazy loading.

public class Movie
{
    public int Id { get; set; }
    public int ThumbnailId { get; set; }
    public virtual Thumbnail Thumbnail { get; set; }

    public string Name { get; set; }
    public double Length { get; set; }
    public DateTime ReleaseDate { get; set; }
    //etc...
}

public class Thumbnail
{
    public int Id { get; set; }
    public byte[] Data { get; set; }
}

public class MovieViewModel
{
    private readonly Movie _movie;

    public MovieViewModel(Movie movie)
    {
        _movieModel = movieModel;
    }

    public byte[] Thumbnail { get { return _movie.Thumbnail.Data; } }
}

Now the thumbnail data will only be loaded from the database when Thumbnail property of the ViewModel is accessed by the UI.

Answer 3

I was having issues with the answer when applied to a ListBox whose ItemsSource is dynamic. In that case, when the source is modified, the ScrollViewer is not necessarily, the trigger is not fired and the images are not loaded.

My main issue concerns the lazy loading lots of highres images in a UniformGrid (which is not virtualized).

To overcome this, I applied a Behavior on ListBoxItem. I find it is a good solution too because you do not have to subclass the ScrollViewer and change the ListBox's Template but only ListBoxItem's.

Add a behavior to your project :

namespace behaviors
{
    using System.Windows;
    using System.Windows.Controls;
    using System.Windows.Interactivity;
    using System.Windows.Media;

    public class ListBoxItemIsVisibleBehavior : Behavior<ListBoxItem>
    {
        public static readonly DependencyProperty IsInViewportProperty = DependencyProperty.RegisterAttached("IsInViewport", typeof(bool), typeof(ListBoxItemIsVisibleBehavior));

        public static bool GetIsInViewport(UIElement element)
        {
            return (bool)element.GetValue(IsInViewportProperty);
        }

        public static void SetIsInViewport(UIElement element, bool value)
        {
            element.SetValue(IsInViewportProperty, value);
        }

        protected override void OnAttached()
        {
            base.OnAttached();

            try
            {
                this.AssociatedObject.LayoutUpdated += this.AssociatedObject_LayoutUpdated;
            }
            catch { }
        }

        protected override void OnDetaching()
        {
            try
            {
                this.AssociatedObject.LayoutUpdated -= this.AssociatedObject_LayoutUpdated;
            }
            catch { }

            base.OnDetaching();
        }

        private void AssociatedObject_LayoutUpdated(object sender, System.EventArgs e)
        {
            if (this.AssociatedObject.IsVisible == false)
            {
                SetIsInViewport(this.AssociatedObject, false);
                return;
            }

            var container = WpfExtensions.FindParent<ListBox>(this.AssociatedObject);
            if (container == null)
            {
                return;
            }

            var visible = this.IsVisibleToUser(this.AssociatedObject, container) == true;
            SetIsInViewport(this.AssociatedObject, visible);
        }

        private bool IsVisibleToUser(FrameworkElement element, FrameworkElement container)
        {
            if (element.IsVisible == false)
            {
                return false;
            }

            GeneralTransform transform = element.TransformToAncestor(container);
            Rect bounds = transform.TransformBounds(new Rect(0.0, 0.0, element.ActualWidth, element.ActualHeight));
            Rect viewport = new Rect(0.0, 0.0, container.ActualWidth, container.ActualHeight);
            return viewport.IntersectsWith(bounds);
        }
    }
}

Then you would have to use this answer in order to add a behavior to your ListBoxItem style : How to add a Blend Behavior in a Style Setter

This leads to add a helper in your project :

public class Behaviors : List<Behavior>
{
}

public static class SupplementaryInteraction
{
    public static Behaviors GetBehaviors(DependencyObject obj)
    {
        return (Behaviors)obj.GetValue(BehaviorsProperty);
    }

    public static void SetBehaviors(DependencyObject obj, Behaviors value)
    {
        obj.SetValue(BehaviorsProperty, value);
    }

    public static readonly DependencyProperty BehaviorsProperty =
        DependencyProperty.RegisterAttached("Behaviors", typeof(Behaviors), typeof(SupplementaryInteraction), new UIPropertyMetadata(null, OnPropertyBehaviorsChanged));

    private static void OnPropertyBehaviorsChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
    {
        var behaviors = Interaction.GetBehaviors(d);
        foreach (var behavior in e.NewValue as Behaviors) behaviors.Add(behavior);
    }
}

Then add the behavior in a resource somewhere in your control :

<UserControl.Resources>
    <behaviors:Behaviors x:Key="behaviors" x:Shared="False">
        <behaviors:ListBoxItemIsVisibleBehavior />
    </behaviors:Behaviors>
</UserControl.Resources>

And a reference of this resource and a trigger to the style of your ListBoxItem :

<Style x:Key="_ListBoxItemStyle" TargetType="ListBoxItem">
    <Setter Property="behaviors:SupplementaryInteraction.Behaviors" Value="{StaticResource behaviors}"/>
    <Setter Property="Template">
        <Setter.Value>
            <ControlTemplate TargetType="ListBoxItem">
                <StackPanel d:DataContext="{d:DesignInstance my:MovieViewModel}">
                    <Image x:Name="Thumbnail" Stretch="Fill" Width="100" Height="100" />
                    <TextBlock Text="{Binding Name}" />
                </StackPanel>

                <ControlTemplate.Triggers>
                    <DataTrigger Binding="{Binding Path=(behaviors:ListBoxItemIsVisibleBehavior.IsInViewport), RelativeSource={RelativeSource Self}}"
                                 Value="True">
                        <Setter TargetName="Thumbnail"
                                Property="Source"
                                Value="{Binding Thumbnail}" />
                    </DataTrigger>
                </ControlTemplate.Triggers>
            </ControlTemplate>
        </Setter.Value>
    </Setter>
</Style>

And reference the style in your ListBox :

<ListBox ItemsSource="{Binding Movies}"
         Style="{StaticResource _ListBoxItemStyle}">
</ListBox>  

In the case of a UniformGrid :

<ListBox ItemsSource="{Binding Movies}"
         Style="{StaticResource _ListBoxItemStyle}">
    <ListBox.ItemsPanel>
        <ItemsPanelTemplate>
            <UniformGrid Columns="5" />
        </ItemsPanelTemplate>
    </ListBox.ItemsPanel>
</ListBox>  

Answer 4

Do you want to load all the images at once? (I recon not all 4000+ movie thumbnails will be shown on screen at the same time). The easiest way I think you could achieve this is to load the images when needed (e.g. only load the ones that are showing and dispose of these (to conserve memory) when not showing).

This should speed up things since you only have to instantiate the objects (and let the ObservableCollection points to the memory address of the objects) and again only load images when needed.

hints to an answer are:

Divide the screen in blocks (pages)
And upon changing the index load the new images (you've already got an observable collection list)

If you still run into difficulties I'll try to give a more clear answer :)

Goodluck

Answer 5

It's taking that long because it's putting everything you've loaded into EF's Change Tracker. That's all 4000 records being tracked in memory, which is understandably causing your app to slow down. If you're not actually doing any editing on the page, I suggest that you use .AsNoTracking() when you grab the Movies.

As so:

var allMovies = MyDbContext.Movies.AsNoTracking();
foreach (var movie in allMovies)
     MovieVms.Add(new MovieViewModel(movie));

MSDN link on it can be found here.