6

I was experimenting with the new core data API NSPersistentContainer and was under the impression that an internal queueing mechanism would prevent write transactions from evaluating concurrently, as detailed in this stack overflow answer NSPersistentContainer concurrency for saving to core data

The way that a lot of pros have been dealing with the problem for a long time (even before NSPersistentContainer did it) was to have an operation queue to queue the writes so there is only one write going on at a time, and have another context on the main thread only for reads. This way you never get any merge conflicts. (see https://vimeo.com/89370886 for a great explanation on this setup which is now what NSPersistentContainer does internally). When you call performBackgroundTask the persistentContainer enqueues that block into an internal serial queue. This ensure that there are no mergeConflicts.

However if I insert multiple entities which share a relationship destination in a tight loop using performBackgroundTask for each iteration, I get a NSMergeConflict error when I save the context:

            let bundlePath = Bundle.main.resourceURL!

        let directoryEnumerator = FileManager.default.enumerator(at: bundlePath, includingPropertiesForKeys: [URLResourceKey.isDirectoryKey, URLResourceKey.nameKey])
        while let url = directoryEnumerator?.nextObject() as? URL {

            if url.pathExtension == "jpeg" {
                let imageData = try! Data(contentsOf: url)

                DataManager.persistentContainer.performBackgroundTask { (context) in

                    //          context.mergePolicy = NSMergePolicy.overwrite

                    let new = Photo(context: context)
                    new.name = url.lastPathComponent
                    new.data = imageData as NSData

                    let corresponding = try! context.existingObject(with: DataManager.rootFolder.objectID) as! Folder
                    new.parent = corresponding

                    try! context.save()
                }
            }

I posted a sample project on github to demonstrate the problem: https://github.com/MaximeBoulat/NSPersistentContainer_Merge_Conflict

The crash seems to be happening because multiple entities are setting their "parent" relationship concurrently, for the same parent, which causes the parent's "children" relationship to be desynchronized across concurrent updates.

This happens even if I set the .automaticallyMergesChangesFromParent property of the incoming context to true. I can prevent the crash by defining the incoming context's merge policy, but that's not an acceptable solution.

Is there any way to configure NSPersistentContainer to properly serialize updates dispatched using the performBackgroundTask API. Or is there something I am missing which is causing these updates to conflict with each other? Or did Apple provide the NSPersistentContainer stack with the expectations that any conflicts encountered when evaluating logic passed into performBackgroundTask should either be fatal, or be disregarded?

Community
  • 1
  • 1
Max Boulat
  • 115
  • 7

3 Answers3

3

I wrote the answer that you are quoting. I was wrong. I have updated it.

I have found that NSPersistentContainer's performBackgroundTask does not have a functional internal queue and it can lead to merge conflicts. When I initially tested it, it seemed like it did, but I found out like you that there can be conflicts. Luckily it is not that hard to fix by creating your own queue. I know it seems strange for Apple to release something that is so broken, but that seems to be the case.

I am sorry for posted incorrect information.

Jon Rose
  • 8,373
  • 1
  • 30
  • 36
  • 2
    It's not broken, this is by design. Just hang on to and use a single background context if you want this behaviour. – trapper Mar 16 '18 at 01:52
2

From the documentation on performBackgroundTask(:):

Each time this method is invoked, the persistent container creates a new NSManaged​Object​Context with the concurrency​Type set to private​Queue​Concurrency​Type. The persistent container then executes the passed in block against that newly created context on the context’s private queue

So, I don't think that's doing what you want it to do. I think you want to call newBackgroundContext(), store it in a property somewhere, and use performBlock(:) on it whenever you want that serialization.

Dave Weston
  • 6,527
  • 1
  • 29
  • 44
  • Thanks. If I have to maintain my own private queue (in the form of a background context variable), it kind of defeats the purpose of encapsulating all the Core Data concurrency logic into a NSPersistentContainer. I am trying to figure out wether there is any way to obtain the same effect as with custom Core Data engines (my version of it is [here](https://github.com/MaximeBoulat/multi-threaded-core-data-swift)) with NSPersistentContainer out of the box without incurring merge conflicts, or wether Apple just expects us to live with merge conflicts when doing background Core Data transactions. – Max Boulat Apr 05 '17 at 17:28
  • 1
    It does seem kind of strange that the `NSPersistentContainer` doesn't attempt to resolve that issue. I can understand that you may want to have the ability to update multiple background contexts at the same time, but making the default of only having one background context for updates so that it's always serialized and you don't have to worry about conflicts seems like a no-brainer (for newbies especially). – Dave Weston Apr 06 '17 at 00:42
  • 1
    This is by design. If it always gave you the same background context then you could do nothing in parallel, one long running background operation would block another completely unrelated background operation. If you need a serialised queue of updates then hang onto and use a single context, or alternatively just maintain your own queue. – trapper Mar 16 '18 at 01:48
1

There’s a bug in your code, line 77 of your DataManager is on a background thread and calls rootFolder which then uses the viewContext. You should not use the viewContext on a background thread. You need to already have the objectID before the background thread begins, you could move the rootFolder.objectID above the block.

malhal
  • 26,330
  • 7
  • 115
  • 133