22

I am starting to setup the core data model for a large scale app, and was hoping for some feedback on proper synchronization methods/techniques when it comes to server database and offline capabilities.

I use PHP and mySQL for my web server / database.

I already know how to connect, receive data, store to core data, etc etc. I am looking more for help with the methodologies and particular instances of tracking data changes to:

A) Ensure the app and server are in sync during online and offline use (i.e. offline activity will get pushed up once back online). B) Optimize the speed of saving data to the app.

My main questions are:

What is the best way to check what new/updated data in the app still needs to be synchronized (after offline use)?

(i.e. In all my Core Data Entities I put a 'isSynchronized' attribute of BOOL type. Then update to 'YES' once successfully submitted and response is sent back from server). Is this the best way?

What is the best way to optimize speed of saving data from server to core data?

(i.e. How can I only update data in Core Data that is older than what is on server database without iterating through each entity and just updating every single time)? Is it possible without adding a server database column for tracking update timestamps to EVERY table?

Again, I already know how to download data and store it to Core Data, I am just looking for some help with best practices in ensuring synchronization across app and server databases while ensuring optimized processing time.

Charles
  • 50,943
  • 13
  • 104
  • 142
JimmyJammed
  • 9,598
  • 19
  • 79
  • 146
  • see http://stackoverflow.com/questions/413086/client-server-synchronization-pattern-algorithm – danh Mar 22 '13 at 02:33

1 Answers1

9

I store a last modified timestamp in the database on both the core data records on the phone, and the mysql tables on the server.

The phone searches for everything that has changed since the last sync and sends it up to the server along with a timestamp of the last sync, and the server responds with everything that has changed on it's end since the provided sync timestamp.

Performance is an issue when a lot of records have changed. I do the sync on a background NSOpeartion which has it's own managed object context. When the background thread has finished making changes to it's managed object context, there is an API for merging all of the changes into the main thread's managed object context - which can be configured to simply throw away all the changes if there are any conflicts caused by the user changing data while the sync is going on. In that case, I just wait a few seconds and then try doing a sync again.

On older hardware even after many optimisations it was necessary to abort the sync entirely if the user starts doing stuff in the app. It was simply using too many system resources. I think more modern iOS devices are probably fast enough you don't need to do that anymore.

(by the way, when I said "a lot of records have changed" I meant 30,000 or so rows being updated or inserted on the phone)

Abhi Beckert
  • 32,787
  • 12
  • 83
  • 110
  • Ditto. I also use a modification date in each of my entities in which I store the time of the last successful server call. On each API call, I pass this to the server, which has triggers that set a similar column in each table any time something changes. And I use a three-context model like in [this question](http://stackoverflow.com/questions/10542097/implementing-fast-and-efficient-core-data-import-on-ios-5) to handle updates asynchronously. – enjayem Mar 22 '13 at 03:44
  • I am also managing the synchronization using last synced data and datetimestamps of table records. – Durai Amuthan.H Nov 22 '13 at 05:01
  • 1
    How can you make sure that the phone's timestamp and server's timestamp using exactly same time. For example if phone's timestamp is 10 seconds earlier than server and there is a change between that time, the data will be lost – Zoon Nooz May 21 '14 at 05:22
  • @ZoonNooz if someone's clock is set wrong, it will screw up, and when they complain of data loss our tech support team will tell them to enable automatic date/time synchronisation which should mean they are accurate to within a split second. We also keep extensive logs on the server, so if data is lost it can manually be recovered (at a billable rate of $100/hour...). I've also got the server set up to assume a 5 minute time delay, where anything on the phone that is less than 5 minutes older than the server is considered to be newer than the server's data. – Abhi Beckert May 21 '14 at 05:27
  • @ZoonNooz if I could do it again, I would also have the phone send the current date/time on it's clock as part of every request. And if it's wrong by more than a few minutes I would fail with an error message telling the user to fix their clock. You might not want to do this check in any request where the phone has to upload a significant amount of data (photos/etc might take minutes to upload). – Abhi Beckert May 21 '14 at 05:27