core data object insert in loop causes problems with NSNumber* property on object novel in inner-loop

Question

I have a loop that has degrading performance as it proceeds (the first loop might be 3 seconds, the last might be 400 over 500 iterations). I've narrowed it down one line of code . . . attaching a unique NSNumber value to a core data object. The catalog size is typically 20K-150K.

Here is the excerpt that runs in an outer loop. The outer loop gets slower and slower each time due to one line of code:

for(int i = 0 ; i < 500 ; i++){
  NSString *key = @"somerandomstring" ; 
  NSNumber* sampleIdNumber = [NSNumber numberWithInt:13] ;
  for (it = catalog.begin(); it != catalog.end(); it++) {
    // it->second->id changes each time in the inner-loop
    NSNumber* tagNumber = [NSNumber numberWithInt:it->second->id];
    DatumMO *newDatumMO = [NSEntityDescription insertNewObjectForEntityForName:@"Datum" inManagedObjectContext:moc];

    // **TagNumber changes each inner loop**
    // **Each time outer loop is calculated it gets slower**
    // **This does not happen if tagNumber is constant**
    newDatumMO.tagId = tagNumber ;

    newDatumMO.name = key;
    newDatumMO.sampleId = sampleIdNumber;
 }
}

A few other points: ARC code, autoreleasepool defined around loop, but doesn't make any difference. 'tagId' and 'sampleId' are both Integer32 in my core data datamodel.

What I've tried thus far: - redefine tagId to int32_t (no change) - use STL map to predefine NSNumber* and do a lookup (no change) - assign tagId constant (fixes problem, but not right) - assign a random int tagId (make it worse, of course)

This seems to be close, but it is unclear if this is the actual problem or if this is currently the best practice for handling scalars in core-data.

http://blog.lorentey.hu/2010/10/31/core-data-scalars/

UPDATE 1 I implemented the scalar model described in the above link with no change in behavior. Going to run further instrumentation on it. The CPU gets starved (is pegged initially) and memory is constant, so either it is swapping out / loading stuff into memory more than it needs to, or writes to core-data slowing it down. Adjusting the core-data write frequency yields no difference in this effect (though obviously effects performance). Removing the autoreleasepool calls also yields obvious effects in this process. Going to try switching to use NSString* next as I'm out of ideas.

Answer 1

Duh, DatumMO.tagId is an indexed column (as is sampleId). Obviously if I'm creating 100K unique, indexed inserts appears to cause significant slow-down as the number of inserts goes up (must be doing a linear scan somewhere?). Not sure if it will provide sufficient speedup or will need to do a post-load indexing, custom or otherwise, but that is another question.