0

We have an iOS application that uses Core Data to persist records fetched from a private web API. One of our API requests fetches a list of Project records, each of which has multiple associated Location records. ObjectMapper is used to deserialize the JSON response, and we have a custom transformer that assigns the nested Location attributes to a Core Data association on the Project entity.

The relevant part of the code looks like this. It's executed within a PromiseKit promise (hence the seal), and we save first to a background context and then propagate to the main context that gets used on the UI thread. 

WNManagedObjectController.backgroundContext.perform {
    let project = Mapper<Project>().map(JSONObject: JSON(json).object)!

    try! WNManagedObjectController.backgroundContext.save()

    WNManagedObjectController.managedContext.performAndWait {
        do {
            try WNManagedObjectController.managedContext.save()
            seal.fulfill(project.objectID)
        } catch {
            seal.reject(error)
        }
    }
}

The problem we're having is that this insert process is saving each Location record to the database twice. Strangely, the duplicated Location records don't have any association with their parent Project record. That is to say, if Location records are looked up with an NSFetchRequest, or if I run a query on the underlying SQLite database, I can see that there are two entries for each Location, but project.locations only returns one copy of each Location. The same (or very similar) process applied to other record types with the same structure also results in duplicates.

I've tried several things so far to narrow down the problem:

  • Inspected the API JSON - no duplicates.
  • Inspected the state of the project.locations property immediately before the Core Data write. No duplicate records are present prior to the objects being persisted, indicating that the deserializer and custom nested attributes transformer are working correctly.
  • Removed the block that propagates the changes to the main thread managed object context, in case this was causing the insert to occur twice. Still get duplicates with solely the write to the background context.
  • Run the app with com.apple.CoreData.ConcurrencyDebug 1 set. No exception is thrown in this process, confirming that it's not a thread safety issue of some kind.
  • Run the app with com.apple.CoreData.SQLDebug 1 set. I can see in the logs that Core Data is inserting exactly twice as many Location rows as expected into the underlying SQLite database.
  • Implemented a uniqueness constraint on the entity. This fixes the problem in terms of what data gets persisted, but will still throw an error unless an NSMergePolicy is set.

The last item in that list effectively solves the problem, but it's treating the symptom, not the cause. Data integrity is important for our application, and I'm looking to understand what the underlying problem might be, or other options I might pursue for investigating it further.

Thanks!

Arvoreniad
  • 510
  • 5
  • 10
  • Does the relationship have an inverse? – pbasdf Apr 08 '20 at 15:39
  • @pbasdf yes, the relationship does have an inverse. – Arvoreniad Apr 08 '20 at 16:32
  • 1
    I would try overriding awakeFromInsert for the Location class, put in a breakpoint, and see if the backtrace gives you any clues as to what code is calling the insert. – pbasdf Apr 09 '20 at 07:52
  • @pbasdf Thanks for the suggestion! I tried this, and `awakeFromInsert` is indeed being called twice for each `Location` - once for the background context, from within the `Location` constructor, and then again when the changes are propagated to the parent context. [This question](https://stackoverflow.com/questions/19856122/why-is-awakefrominsert-called-twice) suggests that's normal though, so I'm not sure if there's anything to be made of the repeated call. – Arvoreniad Apr 09 '20 at 16:02
  • So, two calls to awakefrominsert for each Location - but still two Location objects (so 4 awakefrominsert calls) for each location in the JSON. Does the backtrace give any information as to the code that led to the inserts? – pbasdf Apr 09 '20 at 20:11
  • @pbasdf Actually only two `awakeFromInsert` calls for each location in the JSON, sorry if I was unclear. The duplicate `Location` objects don't manifest themselves right away - I can see them in the underlying SQLite database after the context gets persisted, and then subsequent `NSFetchRequest`s return duplicate location objects. But immediately after the initial insert into the context, before persistence, there doesn't appear to be any duplication. – Arvoreniad Apr 09 '20 at 20:30
  • Curiouser and curiouser... so in your code above the duplication happens not in the Mapper.map call, but after the context save? – pbasdf Apr 09 '20 at 20:53
  • @pbasdf that seems to be the case. Neither `awakeFromInsert` nor the contents of `project.locations` indicate any duplication from ObjectMapper. – Arvoreniad Apr 09 '20 at 22:45
  • Clutching at straws: anything in `willSave` for Project or Location that might generate the extra objects when the save is triggered? – pbasdf Apr 12 '20 at 08:05
  • @pbasdf I'm afraid not, I'm not making use of `willSave` at all. Thank you for your efforts on this, I appreciate it. – Arvoreniad Apr 13 '20 at 19:56

1 Answers1

0

A year and eight months later, I finally got to the bottom of this bug when a similar issue occurred with a different set of records. The problem was that I was calling ObjectMapper on each Location object twice. I was using ObjectMapper's mapArray method within a custom ObjectMapper TransformType to deserialize and persist the Location records associated with each Project, which worked as follows:

let locations = Mapper<Location>().mapArray(JSONObject: value as AnyObject)

However, what I had overlooked is that I was also overriding the constructor for Location and calling ObjectMapper again there:

required public init?(map: Map) {
    let entity = NSEntityDescription.entity(forEntityName: "Location", in: WNManagedObjectController.backgroundContext)
    super.init(entity: entity!, insertInto: WNManagedObjectController.backgroundContext)
    mapping(map: map)
}

The line mapping(map: map) was unnecessary, and proved to be the culprit. In a similar scenario with two levels of one-to-many associations, this had the somewhat amusing consequence of quadrupling (!) the records at the second level - their parents had been duplicated, each copy of which subsequently duplicated its children. This was what ultimately led me to the cause of the bug.

Arvoreniad
  • 510
  • 5
  • 10