1

I am building a project where I need to prepopulate a coredata database with existing data.

I built a parser to create the sqlite file in the iOS Simulator, everything works fine. I am using a single entity, and one of the attribute is indexed. Performance after parsing my data file into core data is great, everything is good.

Now I am using the generated sqlite file (~200Mb) in project with same data model, same index, etc... and on first startup I copy over the db file to prepopulate the data

    NSString *defaultStorePath = [[NSBundle mainBundle] pathForResource:@"myproject" ofType:@"sqlite"];
    NSString *storePath = [[[self applicationDocumentsDirectory] path] stringByAppendingPathComponent: @"myproject.sqlite"];

    NSError *error;
    if (![[NSFileManager defaultManager] fileExistsAtPath:storePath]) 
    {
        if ([[NSFileManager defaultManager] copyItemAtPath:defaultStorePath toPath:storePath error:&error])
            NSLog(@"Copied starting data to %@", storePath);
        else 
            NSLog(@"Error copying default DB to %@ (%@)", storePath, error);
    }

The copying works fine, and the data can be accessed normally. However the performance is terrible, and the index is clearly not being used.
A look at the size of the sqlite file after the copy operation, it went from 200Mb to 120Mb.
Everything looks alright in the model, what needs to be indexed is checked as indexed.

1) Is there a way for the index data not to be removed when copying the sqlite over?
2) Is it possible to programmatically rebuild the index?
3) Any other thoughts?

JP Hribovsek
  • 6,707
  • 2
  • 21
  • 26
  • Did you ever figure out if you can programmatically rebuild the index? I have the same problem as you. Thanks! – baselq Jul 20 '12 at 01:07

2 Answers2

1

Check at the Apple documentation about this issue:

Although Core Data supports SQLite as one of its persistent store types, the database format is private. You cannot create a SQLite database using native SQLite API and use it directly with Core Data (nor should you manipulate an existing Core Data SQLite store using native SQLite API). If you have an existing SQLite database, you need to import it into a Core Data store (see “Efficiently Importing Data”).

To sum up, don't do it. The database schema is private and may change.

I use CSV files to pre-load all my initial data to CoreData on background when the application starts for the first time. Beware of multithreading CoreData access, by the way.

Hope it helps.

redent84
  • 18,901
  • 4
  • 62
  • 85
  • ummm, no I am afraid it's not going to help in this case. I am not using any SQLite API btw, just copy/pasting the sqlite file, which works just fine if there wasn't that index problem. My data comes from a 2 million lines csv file (and that is just a test, eventually it would be closer to 10 millions), so loading in the background at first start is not gonna work, the app would need a few hours of loading time at startup... – JP Hribovsek Apr 30 '12 at 22:36
  • You cannot rely on copying the SQLite database for CoreData, the schema may (and does) change between devices and iOS versions. – redent84 Apr 30 '12 at 22:42
  • well, in this case, it is the same app, the same device (simulator)... I just do parsing in a first pass, then remove the sqlite file and place it in the bundle. In the second pass, I copy the sqlite file at first start, again this works fine, just the index goes away. I do understand the point of not having this supported. The question then is whether or not the index can be rebuilt programmatically. Or, what could be the best workaround? I can think of a few apps with massive databases that are clearly indexed, and yet are functional instantly at first start. – JP Hribovsek Apr 30 '12 at 22:52
0

Looks like the problem was that the project didn't get cleaned well in between tests, it could have been a bug in Xcode 4.3 I was using at the time.

The same methodology is working fine now.

JP Hribovsek
  • 6,707
  • 2
  • 21
  • 26