0

I am using LiteDB to store approx.1 million records. When i retrieve records from LiteDb after filtering and convert .ToList() from IQueryable, it is very slow even the result of the query is just one record. I am really stuck on this.

I found that the problem with the convert .ToList().

I was reading some about similar kind of problem (i.e .ToList()) already faced. But not getting any correct solution.

my sample code:

IQueryable<student> activeFilterResult = liteDatabase
   .GetCollection<student>(studentcollection)
   .Find(Query.All())                                                  
   .AsQueryable<student>();

...............

activeFilterResult  
  .OrderBy(studentrecord => studentrecord.Id)
  .Select(studentrecord => studentrecord.Id)
  .Distinct()
  .ToList();
Daniel A. White
  • 187,200
  • 47
  • 362
  • 445
Subramanian
  • 1
  • If they are all the same you still have ordered 1M records before discarding them. ToList creates a copy but only of what it gets fed. – TaW Oct 09 '19 at 07:36
  • What if you remove `Distinct`? – ProgrammingLlama Oct 09 '19 at 07:37
  • 1
    Possible duplicate of [Is there a performance impact when calling ToList()?](https://stackoverflow.com/questions/15516462/is-there-a-performance-impact-when-calling-tolist) – jazb Oct 09 '19 at 07:40
  • It would be awesome if you could share a [mcve]. – mjwills Oct 09 '19 at 08:14
  • 3
    Before the call to `ToList`, all you do is basically construct a query. No data is actually retrieved or ordered or checked for duplicates, yet. You just planned a workflow to be executed at some point in the future. By calling `ToList`, all those planned things now actually need to happen. It's not the "fault" of `ToList` that this is slow, it's because you have planned a lot of stuff to do. – Corak Oct 09 '19 at 08:25
  • Looks like you are trying to get all entities stored in database. In that case it's obvious it will take some time. In order to mitigate that impact a bit you can OrderBy() and Distinct() in memory, that is after you call ToList(). – Arthur Oct 09 '19 at 08:33
  • 2
    @Arthur - *usually* the DB is "better" at stuff like ordering or looking for duplicates. I never worked with LiteDB, so maybe it's really better to simply get everything and then work on it in memory. – Corak Oct 09 '19 at 08:37

1 Answers1

0

dont use ToList() without any filter for all data to output. 1.some filter like First() or Take(20) 2.than....call ToList()

sgf
  • 11
  • 2
  • 1
    As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Dec 09 '21 at 16:16