8

I'm executing a fetch request that returns about 2000 entities. Currently, this takes about 20 seconds on my device. So I thought I might set a fetch limit of 100, and then when the user scrolls to the end of the table view, fetch the next 100 entities. This can be accomplished using NSFetchRequest's setFetchLimit and setFetchOffset.

However, what I can't figure out is, if on my second fetch where I'm fetching objects 101-200, what would happen to the objects 1-100? Would I have to use separate NSFetchedResultsController for every 100 items, and then configure my table view data source methods to query based on multiple fetch results controllers? Or can I somehow use the same NSFetchedResultsController to somehow fetch 100 entities at a time, but upon every subsequent fetch, just add the next 100 items to the original 100 items?

Edit: Here's my code:

    NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];

    NSEntityDescription *entity = [NSEntityDescription 
                                   entityForName:@"MessageObject" inManagedObjectContext:appDelegate.managedObjectContext];

    [fetchRequest setEntity:entity];
    NSPredicate *predicate= [NSPredicate predicateWithFormat:@"ANY tags.tagName==%@", currentTagObject.tagName];
    [fetchRequest setPredicate:predicate];
    NSSortDescriptor *sort= [[NSSortDescriptor alloc] initWithKey:@"createDate" ascending:NO selector:@selector(compare:)];
    [fetchRequest setSortDescriptors:[NSArray arrayWithObject:sort]];
    [fetchRequest setFetchBatchSize:5];
Snowman
  • 31,411
  • 46
  • 180
  • 303
  • Can you show your construction of the fetch request? As far as I know, fetched results controllers (and I think simple fetch requests, too) use faulting to do this for you. It's possible that your "query" can be optimized. – Chris Trahey Jun 22 '12 at 23:23
  • I've posted some of my code. I don't understand - why does it take 20 seconds to fetch the entities then? When I set a limit of 100, it fetches in 1 second, but without a limit, which fetches 2000 entities, it takes 20 seconds. – Snowman Jun 22 '12 at 23:27

2 Answers2

4

You don't actually need to do anything to achieve the fetching in batches of the size you've specified. You will get the behavior you're describing simply be executing a single fetch request on a single controller - by setting the batch size property, you're simply hinting to CoreData what you consider to be the optimal batch size by your own consideration. CoreData will handle fetching objects in sequential batches of the size specified as you need them - and it will fault managed objects when necessary for memory consumption.

isaac
  • 4,867
  • 1
  • 21
  • 31
  • Why does it take 20 seconds to fetch the entities then? When I set a limit of 100, it fetches in 1 second, but without a limit, which fetches 2000 entities, it takes 20 seconds. – Snowman Jun 22 '12 at 23:28
  • Well, it still performs the fetches based on the size you specify. Fetching 100 is faster than fetching 2000. On your fetchRequest, set the batch size to 50 or 100. That's it. There's nothing else to it. All your records will continue to load as needed, because CoreData handles the subsequent necessary fetches and faults. – isaac Jun 22 '12 at 23:31
  • But so this is not the solution - I need to eliminate the 20 second wait, and setting the batch size doesn't appear to make a difference. Also, what's weird is that I just set the batchSize to 2000, and it fetched in 9 seconds, compared to a batchSize of 5 which took 20 seconds. What's up with that? – Snowman Jun 22 '12 at 23:34
  • So you're fetching 2000 entities, but how many records are you searching on? It sounds like the fetch is taking an inordinate amount of time, which is most likely one of a few things: There's something wrong with the managed object model, you're not phrasing your query properly, or you're trying to search too large a set (which basically means you need to reconsider your query and search architecture). – isaac Jun 22 '12 at 23:40
  • I only have a total of 2100 entities, so that's how many it's searching on. And my query is posted above, it's just a typical query where I'm searching the MessageObject's many-to-many Tags entity, to see if it has a tag object with the current tag object name. – Snowman Jun 22 '12 at 23:43
  • So with a batchSize of 0, which is unlimited I guess, actual fetching time is 5 seconds, and it takes another 4 seconds to update the tableView. – Snowman Jun 22 '12 at 23:46
  • I tried commenting out the setPredicate, so that it would just fetch all entities, and it made no difference in time. – Snowman Jun 23 '12 at 00:12
  • 1
    What about commenting out the sort? Even if you only get the first 100 records, you still have to sort them all to know which ones to return. It could be taking most of it's time here. – lnafziger Jun 24 '12 at 01:03
3

Using ANY is a pretty expensive operation. I'm not sure about the relationships, but you Could try fetching your tags entity for the specific tag name rather than on your mesage object entity. Then use the inverse relationship to get the message object entities

Also your sort is using a comparator which I'm sure is not helping matters. The date should be stored by core data as an integer number so you can just do a straight sort without using a selector.

Core Data is VERY fast, I've done a keyword search through 65,000 records and have had returned results in milliseconds.

Good luc

timthetoolman
  • 4,613
  • 1
  • 22
  • 22
  • 3
    It's never about answers, just about guiding people in the right direction. – Snowman Jun 24 '12 at 03:50
  • @tim that's actually a good idea, about retrieving the single tags object and get the message set. Would be much faster. However, how can I tie that up to a NSFetchedResultsController? If a tag has 2000 objects, I don't want to load them all into memory at once. With an fetch results controller, only a handful of them would be loaded into memory at any given time.. – Snowman Jun 24 '12 at 04:19
  • @mohabitar You should not have to use a separate NSFetchedResultsController. It provides a mechanism for refetching what you need automatically, based upon the batch size of the fetch request. For example, if you can only show 10 records on your screen, you set the batch size to 25, and it will fetch in more as you scroll. – timthetoolman Jun 24 '12 at 06:20