Save large amount of Data into CoreData

Question

I want to save the large amount of data into CoreData, The resource of my Data is in CSV format.

I know the following ways to achieve it.

1. With Help of Multiple MOC - I already tried with this one but its only help me to not freeze the my app on Main thread, Actually after the splash I want to use that data as well and while data is being save at background thread - Chances to not have that data immediately on required screen(Actually even I faced it, I was not able to fetch entity I want - Perhaps still its remaining to save on CoreData stack?)
2. Do I need to convert that file into plist extension file? Is that the good idea to store large amount of data in plist file? And unfortunately I was not able to find any online tool which converts my .csv file to .plist file? I guess reading the data from plist is not a big deal.

3. Preloaded database need to be import into CoreData, Right now I am working by this way, I successfully converted my CSV file to SQLite file and imported it to bundle and then copy it from the bundle to Document Directory while NSPersistentContainer will be initialize. Now I have some few queries on this point as below.

   • I found the SQLite file inside Document Directory, But its actually just copy the Existing SQLite's Tables with My CoreData Entities, What I want is that One of Table from Existing SQLite should be merge with My CoreData Entity, Do I need to loop over the table for each row and then manually save it to Entity? If so then its the same situation as I mentioned at Point 1.?
   • Somewhere I found that CoreData DB structure and SQLite DB structure are different, so What I did - I created some Dummy Data on CoreData and Found the SQLite file, Inside that SQLite file I imported the data from CSV Table and now That SQLite file I have used in application bundle, I guess here I achieved something as I got an error says migration Failed as Table already exist on Database, Don't I need to define Entity first into DataModel?, OR Don't I need to Create the table inside SQLite File?(If so then Where I can pre-populate the Data?)

Please suggest What I Need to Do? And how can I achieve it? Also It would be great if you suggest any other options to Do so.

Answer 1

To extract an entity-aware version of your SQLite database files, you can run your app in a simulator and print out to the console the path to the file generated. Then using Finder retrieve the file after Core Data is done parsing/saving all the records.

You can then bundle up the generated SQLite file in your projects and have it ready to use.

Answer 2

I think it solution is bad. Because you make few problem
1) Your build will be more size - and you don't fix this. And user would be loading app too long time.
2) You will be have sqlite file in bundle application, it means you can't change this Database in runtime.

It can be fix with two way:
1 way - make two Core Data stack one readonly, second read/write with sqlite in application support directory. But it complicates work with the Core Data. And it way available only if your don't need change data how stored in app bundle.
2 way - make copy sqlite file to application directory, and work with this. But you will have two copy of one file.

If you sure in this way
1) You need make simple parser what work only debag mode and simulator.
2) You should setup Core Data with off journal mode (we need only sqlite file)
How to disable WAL journal mode

options["journal_mode"] = "DELETE"

More information about journal mode
https://developer.apple.com/library/archive/qa/qa1809/_index.html

3) Parsing your scv and get sqlite file in Simulator
4) Add sqlite file in your resources

But I recommend make API requests, how send data in your application
1) You can send some hash or last update and receive only diff data (not all)
2) You can ask only data how need at this moment with pagination data and support hash.