0

Background

I have a database table that contains a large amount of records, 500 thousand. I use to load them all at once and binding them to a GridView. Obviously the page loading time is adversely affected by this.

My solution

I thought I could use the Lazy Loading feature of Entity Framework to only load sections of my data that were in view/ being displayed by the gridview. So I implemented a EF solution, but I now get a system.OutOfMemoryException.

After some reading it turns out EF loading all the records twice, which is why I'm obviously running out of memory.

Question

Is EF the right solution for lazy loading large amounts of data? If so how do I go about it?

Cœur
  • 37,241
  • 25
  • 195
  • 267
Dan Cundy
  • 2,649
  • 2
  • 38
  • 65
  • 3
    `Is EF the right solution for lazy loading large amounts of data?` The short answer is that you shouldn't try and display 500,000 records in a view. You should implement some kind of filtering or paging instead. No reasonable person is going to read a grid with 500,000 records in it. – mjwills Oct 27 '17 at 10:30
  • Possible duplicate of [Lazy Loading vs Eager Loading](https://stackoverflow.com/questions/31366236/lazy-loading-vs-eager-loading) – abr Oct 27 '17 at 10:32
  • 2
    Consider to let your GridView allow custom paging. See GridView.AllowCustomPaging, and several examples of custom paging on the internet – Harald Coppoolse Oct 27 '17 at 10:36
  • 1
    Guys, I know the difference between Lazy and Eager Loading. Hence why I went with EF. However EF is still behaving like it is eager loading. – Dan Cundy Oct 27 '17 at 10:44
  • @HaraldCoppoolse, thanks looking into these properties now. – Dan Cundy Oct 27 '17 at 10:45
  • What do you think `Lazy Loading` means? – mjwills Oct 27 '17 at 10:46
  • @mjwills I think this maybe where the confusion lies. I thought it meant, loading only the records needed at any one time. For example in my case, loading only one page's worth of database records at a time. – Dan Cundy Oct 27 '17 at 10:51
  • 1
    `For example in my case, loading only one page's worth of database records at a time.` Alas no, that is not Lazy Loading. – mjwills Oct 27 '17 at 11:32
  • When a GridView is sent to the browser, all links to the database are lost. So that GridView cannot read data from the database "when it comes into view". Instead the GridView reads *all* data, which causes the lazy loading to kick in for an extra db-read (per row) when that data wasn't there yet. – Hans Kesting Oct 27 '17 at 13:29

1 Answers1

1

I think what you want is Paging rather than Lazy loading.

Have a look in this https://learn.microsoft.com/en-us/aspnet/mvc/overview/getting-started/getting-started-with-ef-using-mvc/sorting-filtering-and-paging-with-the-entity-framework-in-an-asp-net-mvc-application

duongthaiha
  • 855
  • 6
  • 17