153

I am developing a mobile web application (for iPhone & Android) with a local database (using html5 webstorage) so my app is still usable when the user is offline.

This is working perfectly, but I want to save the local data on a server. So I need to synchronize the local DB with a DB on a server. The synchronisation can only be one way, but in the future, I would like to synchronize it in the both way (server <-> local DB).

This requierement looks very common (or will be common in the future for mobile web app), but I can't find a library doing that.

I know google is doing that in their mobile web app (ex. gmail), and I found the WSPL project a google project but without source to download.

If I can't find a solution, I will create a library to do that, as one way sync doesn't look difficult, but I wonder if there are other solutions.

Samuel
  • 5,439
  • 6
  • 31
  • 43
  • 2
    I don't know if there are any libs, but easiest way to do this seems to be storing modification timestamp, and transferring changes to records which are newer than the records on other side, and also transferring additions and deletions since last sync. It may go crazy if local and server clocks are not in sync, but you'll think of something. -- Posting as comment since it's probably not very helpful and doesn't provide you with an answer. – Ivan Vučica Jan 07 '10 at 21:16
  • Thanks Ivan. You are right, if the local and server clocks are not in sync, it could be messy... I just found that : http://quickconnect.pbworks.com/Using-Enterprise-Synchronization It says it can synchronize a local HTML 5 DB with a DB in a server. I need to have a deeper look on that, and see if it can run outside the QuickConnect Framework... – Samuel Jan 09 '10 at 14:53
  • 4
    Topics are for *discussions*, Stack Exchange is for *questions*. At one point posts like this were accepted on Stack Exchange, but no longer. – casperOne Mar 04 '13 at 20:43
  • 1
    How about CouchDB? http://couchdb.apache.org/ – julianm Dec 03 '11 at 00:16
  • I don't think it provides WebSQL synchronisation... – Samuel Dec 13 '11 at 00:22
  • I found another solution : http://impel.simulacre.org/blog/transparently_synchronize_html5_databases.html Looks great, but you will need to use Mootools library, and the Impel ORM... – Samuel May 31 '10 at 15:24
  • Microsoft Access has a way of synchronizing rows by using unique hash type ID. That could be an additional layer including a transaction date. If it is just one table this is an easy task. When you get into multiple tables and dependencies, this is when can be messy. – Sun Nov 28 '13 at 18:15
  • You can mark all offline changes or new records that are done in the HTML5 (local) db. The sync process will just effect on synchronizing the changes to server which are marked as "offline". – Mega Apr 29 '14 at 09:05
  • These answers are out of date, check out http://pouchdb.com/ – J Chris A Aug 04 '14 at 21:31
  • @JChrisA Your comment is out of the subject, we are talking about WebSQL DB here – Samuel Aug 13 '14 at 09:11
  • Is there any framework to Sync IndexDB database to DB? – Subin Chalil Dec 09 '15 at 07:19

2 Answers2

70
  • I created a small JS lib named WebSqlSync to synchronize a local WebSql DB with a server (client <-> server). Very easy to use and to integrate in your code :

https://github.com/orbitaloop/WebSqlSync

  • The open source project QuickConnect contains a JS library to synchronize the local HTML5 SQLite DB to a server DB (MySQL or other) :

http://quickconnect.pbworks.com/Using-Enterprise-Synchronization

To use this lib, you need to use the DataAccessObject of the framework to access your DB. It works by storing all the SQL request applied to the DB (except select of course) , and sending them to the server. It's great to manage deletion, but it's a little heavy if you have a lot of updates, and the server need to use the same SQL language...

  • Another project from QuickConnect is a native SQLite synch (in Objective C for iOS or Mac OS and in Java for Android) :

http://www.quickconnectfamily.org/qcdbsync/ (I think it store also the history of all the SQL requests)

  • And i just found another promising JS library : persistenceJS

https://github.com/zefhemel/persistencejs

"persistence.js is an asynchronous Javascript object-relational mapper library. You can use it in the browser, as well on the server (and you can share data models between them)."

They have a DB synch module: DOC of persistence.synch.js

(works with HTML5 DB SQLite or Google Gears on the client, and MySQL on the server)

  • And there is also Impel.inTouch. It looks very easy to use (with php files included), but you must use the Mootools framework in the client side :

http://impel.simulacre.org/api/Impel.inTouch

  • Sencha has also a synchronisation service: Sencha.io. Looks great, but it's dependent of the Sencha Touch framework:

http://www.sencha.com/products/io/

Samuel
  • 5,439
  • 6
  • 31
  • 43
  • Hi Samuel, did the js lib work out for you? – Mathias Conradt Aug 14 '10 at 13:53
  • DB sync is not my priority for now, so I just give up, waiting for a more standard and robust solution... – Samuel Aug 18 '10 at 13:25
  • as the WebSQL Standard is already deprecated ([link](http://www.w3.org/TR/webdatabase/)) and work will not be continued, I would rather search for some alternatives, as WebSQL will probably not be supported in future releases! – Matthias B Nov 07 '11 at 12:42
  • WebSQL is being used in thousands of webapps (even Apple and Google use it), so I don't think it will be removed soon... – Samuel Nov 08 '11 at 14:56
  • 3
    After testing all of them, I think I will develop my own small JS lib to synchronize WebSQL with a server DB. It will be a double synch (local <-> server) and will not have any dependency. I'll post here the link to the code once finished – Samuel Jan 13 '12 at 14:37
  • I had the similar situation to sync mobile data with enterprise database. I have used persistencejs and written custom sync logic based on sync/not-synced row property on server as @samuel mentioned. – dhaval Feb 01 '12 at 11:08
  • 2
    I have commited the first version of my own sync solution named WebSqlSync : https://github.com/orbitaloop/WebSqlSync (cf below answer) – Samuel Feb 02 '12 at 14:56
  • 1
    Hi Guys, I've started a persistencejs plugin for restful synchronization. It still in development, but if anyone wants to check it out: https://github.com/robertokl/persistencejs and a working example on server/client side with ruby on rails: https://github.com/robertokl/persistencejs-restfulSync-example – robertokl Aug 09 '13 at 15:14
  • Do you have anything similar but for localStorage and not WebSQL? –  Apr 24 '14 at 15:21
  • A quite recent alternative is https://replicache.dev/ – thSoft Jun 02 '22 at 20:07
18

I have developed a generic sync solution called WebSqlSync.

It's not dependant of any framework. It's available here : https://github.com/orbitaloop/WebSqlSync

Extract of the README file :

WebSqlSync

Automatically synchronize a local WebSql database (SQLite in the navigator) to a server. (2 way sync : client <-> server)

Very easy to integrate to your existing app and very easy to use (2 functions to call : initSync and syncNow)

Usage

Initialize

You need to initialize the lib (at each startup for example).

It will automatically create 2 tables (if they don't already exists, one to store all the new or modified elements (table new_elem) and one to store the date of the last sync (table sync_info). It will also create SQLite triggers in order to watch the INSERT or UPDATE on the tables you want to synchronize (to automatically insert the modified elements in the new_elem table):

DBSYNC.initSync(TABLES_TO_SYNC, webSqlDb, sync_info, 'http://www.myserver.com', callBackEndInit);

Where TABLES_TO_SYNC is the list of table that you want to sync with the server, ex :

TABLES_TO_SYNC = [
    {tableName : 'table1', idName : 'the_id'},
    {tableName : 'table2'} //if idName not specified, it will assume that it's "id"
];

Synchronize

To start the synchronization, you need to call the syncNow function. You can call it every X seconds, or after some changes for example :

DBSYNC.syncNow(callBackSyncProgress, function(result) {
     if (result.syncOK === true) {
         //Synchronized successfully
     }
});

And that's all you need to do on the client. On the server side, you will need to code your own solution (but it's not complicated). And there are some example inPHP & Java. Again, contributions are welcome.

Samuel
  • 5,439
  • 6
  • 31
  • 43
  • How well would you say this have worked for you, summarizing it one year later? I'm looking for a good client-side DB that works for browsers and mobile units. – Niklas Feb 27 '13 at 09:45
  • 1
    WebSQLSync is working very well with more than 25 apps in production (iOS and Android). WebSQL is really great and fast. It's working on iOS, Android, Blackberry (the latest version I think) and of course chrome and safari. But it's not working on IE and firefox, because the API has been depreciated by W3C.. – Samuel Mar 04 '13 at 20:33
  • Alright, both positives and negatives there. Thanks for the recap! – Niklas Mar 04 '13 at 23:45
  • 2
    Do you have anything similar but for localStorage and not WebSQL? –  Mar 28 '14 at 16:50