How to sync flutter app's offline data with online database efficiently

Question

lets say I have a todo app that stores the data in sqflite database(locally on the phone) when the app goes online I want the data to be synced with my online database say mongodb or firestore.I don't want to do complete overwrites or creating the new table everytime,I am looking for some efficient solution that only updates the changes to the database.

That's an architectural question. Better go another channel. — Hosar, Sep 11 '19 at 04:29
Firestore has already support for offline mode. Why have 2 DB? — Alvin Konda, Sep 11 '19 at 05:48

score 17 · Answer 1 · edited Jun 05 '20 at 15:08

Firebase Cloud Firestore makes all of this for you. If You can't use this, other solution is much more complicated as syncing information can be very tricky.

Imagine the user has 2 devices, both saving the same data, which version should the server keep, the newest created data, or the data the server received first?

SQLite is good for structured data, but you could save your data as a JSON string using System Preferences and just push that JSON to the server, it's a lot simpler if can do it this way.

Again, this really not a simple answer to give and varies from project to project.

score 9 · Answer 2 · edited Jan 01 '22 at 13:37

9

I think this will help to construct offline and online sync：

flutter_offline

http

sqlite

edited Jan 01 '22 at 13:37

Mahesh Jamdade

17,235
8
110
131

answered Nov 13 '19 at 10:06

Lydan

91
1
2

is flutter_offline store data as a local DB? – Didarul Alam Rahat Aug 29 '20 at 16:42
1

It detects connectivity and change widgets accordingly. – tabebqena Jun 05 '21 at 11:05

score 5 · Answer 3 · answered Apr 22 '20 at 03:21

As already pointed out, this is an architectural question and there's probably no simple answer. Here are two main thoughts :

Scenario A : You are just updating the transaction data and sequence doesn't matter e.g. mobile attendance app. The user's location along with date time stamp is recorded on the server. And server does everything else (validation, processing, reporting).

Scenario B : Sequence does matter. If you just keep the offline connectivity then older data may overwrite newer data in the following like scenario : user has two devices. the device having new data comes online first followed by the device with old data

I'd dealt with both of the scenarios and for scenario B, I did the following :

Created a 'queue' table.
All the entries first goes to the queue (online or offline)
The inserts in the queue entry triggers Firebase Cloud function. The function does the validations and processing (example below).
The function makes the entry in the actual table.

Step 3 is particularly useful when working with apps such as Todo. It is because you might want to log the event (say in the log file) but not to udpate the todo item status if older data comes later.

How about saving the data in local storage with date/timestamp and server stores the data by sorting according to timestamp? So even if the user has multiple devices the data would always be in correct sequence. — Aritra Dattagupta, May 31 '20 at 17:03
There are more than one way of using a local storage and keep the data in sync with the server. It depends on the project requirement and the data setup. The important question is - do you want to take a pain and write all the code, test it or use a proven functionality from Google (or like) ! — Sukhi, May 31 '20 at 17:20
Many projects wouldn't want to use Firebase/AppSync because of the related cost. I am using Hasura GraphQL and custom node server on Digitalocean which is way cheaper than AWS or Firebase with equal amount of performance. — Aritra Dattagupta, Jun 01 '20 at 10:18

score 0 · Answer 4 · answered Jul 16 '23 at 16:49

Couchbase Lite is a database that has a mature data sync feature.

It's a document-style NoSQL database, so it allows you to store JSON without having to define a schema.

Documents can be modified locally and synchronized with a remote Couchbase Server database when there is a network connection. Conflicts between changes from multiple devices are detected and can be resolved automatically or through a custom conflict resolver.

The already mentioned Cloud Firestore is not a full offline-first solution and is not suitable for all use cases. It does not allow you to precisely control which data is available locally when the user is offline. It only caches a certain amount of data that has already been requested. Couchbase Lite provides multiple options for controlling how documents are synced with the remote server, including filter functions, lists of document ids and channels. Cloud Firestore also does not have a built-in conflict detection system.

Disclaimer: I am the author of the Couchbase Lite Dart/Flutter SDK.

How to sync flutter app's offline data with online database efficiently

4 Answers4