Lazy loading and grouping in WPF DataGrid

Question

I need to display a list of data (100,000~500,000) in DataGrid, in which several columns display images instead of simple data. I have to use "Lazy loading" because loading all the images at once is impossible.

This list should support filtering, sorting and grouping.

My current implementation is to trigger the "fetching images" by the ScrollChanged event (with the ScrollChangedEventArgs.VerticalOffset as the startIndex). It works fine if the list has not been grouped. But, if I use CollectionView to group the data, how can I know the correct data items which need fetching images when the vertical offset of the scroll bar can not be see as a data index anymore?

I was thinking of another solution by using two ViewModels. One is normal ViewModel, and one is GroupedViewMode. And I can display the grouped data in DataGrid like this:

public class GroupedData{

    public string ColumnName{get;set;}
    public bool Expanded{get;set;}
    public List<Data> List{get;set;}
}

And, then, I can somehow get the data which need fetching images when the scrollbar value is changing. But, here is another question... how can I bind this data format in a DataGrid?

=========================

IsVirtualizingWhenGrouping is good, but it cannot fully solve my problem, I still need to load all the data in the memory in the first place. The data may be up to Gigabytes, it is memory consuming. So my current solution is to load all the data with image excluded to support sorting, grouping etc, then load images when needed.

Your question has a number of issues. The most obvious is this one: *"But, here is another question..."* Please don't write multiple questions into one. Instead, ask a question focused on a single problem. If you have multiple problems, ask multiple questions (better not all at once, but start with the first and try to work with the result before asking the next thing) — grek40, Oct 13 '17 at 07:45
Possible duplicate of [WPF DataGrid Virtualization with Grouping](https://stackoverflow.com/questions/1477044/wpf-datagrid-virtualization-with-grouping) — ASh, Oct 13 '17 at 08:31
@grek40 I'm sorry I have posted a mixture of everything. I am new to WPF. I just hoped maybe someone had faced similar situation before and could give me some hint... — Siamca, Oct 13 '17 at 10:07
@ASh UI virtualization can partly solve my problem, but not all. — Siamca, Oct 13 '17 at 10:10
Where do the images come from? Are they stored in the database, somewhere else or dynamically created from database values? — grek40, Oct 13 '17 at 10:40
Are the data in the file formatted in a way that you can set the read pointer to an arbitrary entry index without reading the entries that come before? (uniform length per entry or some sort of index table) — grek40, Oct 16 '17 at 06:15
@grek40 yes. it's uniform length. So I can set the pointer to the exact entry index. I have set "CanContentScroll = true" and then I can get the index of the first item currently displayed in the screen from ScrollChanged event. But the tricky thing is, once the data is grouped, the scrollbar scroll in pixels, not in logical units anymore...I am considering calculate pixels of datagrid row to get the current index... — Siamca, Oct 17 '17 at 01:22

score 0 · Answer 1 · answered Oct 13 '17 at 09:11

0

Since .Net 4.5, Microsoft has added IsVirtualizingWhenGrouping property. If you set this property of your DataGrid, then you don't need to do it all yourself, as DataGrid will load only the visible data / images.

<DataGrid VirtualizingPanel.IsVirtualizingWhenGrouping="True">

answered Oct 13 '17 at 09:11

Yvonnila

635
1
5
22

This is UI virtualization, it still needs to load all the data in the memory. What I'm trying to do is sort of data virtualization. As far as I know, there is no good solution yet. – Siamca Oct 13 '17 at 09:50

score 0 · Answer 2 · answered Oct 17 '17 at 07:08

I think you have to lazy-load your image in the viewmodel, so it will only be loaded when it is first requested by the (virtualized) UI.

If you create a MemoryMappedFile of your source and create a MemoryMappedViewStream for each data entry*, the Data class could look as follows:

class Data
{
    // Whatever size every data entry may have
    const int SizePerDataInFile = 100;
    // Offset of the image data within a single data entry
    const int ImageDataOffset = 40;
    // Size of the image data
    const int ImageDataLength = 60;

    public Data(MemoryMappedViewStream entriesFile)
    {
        _EntriesFile = entriesFile;
    }
    private MemoryMappedViewStream _EntriesFile;


    public int EntryIndex { get; set; }

    public string GroupableProperty { get; set; }


    public BitmapImage _Picture;
    public BitmapImage Picture
    {
        get
        {
            if (_Picture == null)
            {
                _EntriesFile.Seek(EntryIndex * SizePerDataInFile + ImageDataOffset, SeekOrigin.Begin);
                byte[] imageData = new byte[ImageDataLength];
                _EntriesFile.Read(imageData, 0, ImageDataLength);

                _Picture = new BitmapImage();
                using (var mem = new MemoryStream(imageData))
                {
                    mem.Position = 0;
                    _Picture.BeginInit();
                    _Picture.CreateOptions = BitmapCreateOptions.PreservePixelFormat;
                    _Picture.CacheOption = BitmapCacheOption.OnLoad;
                    _Picture.UriSource = null;
                    _Picture.StreamSource = mem;
                    _Picture.EndInit();
                }
                _Picture.Freeze();
            }
            return _Picture;
        }
    }
}

This would mean you initialize the "cheap" properties for all data and they will be used for grouping etc. The image is only loaded when it is accessed by reading the relevant data block from the file.

* Even though I suggest a MemoryMappedViewStream per data entry, I'm not really sure about that - maybe a shared viewstream for all data is enough.

Lazy loading and grouping in WPF DataGrid

2 Answers2