1

I'm trying to get my head around the use of IndexDB. I have an SQL database which I access via REST and I'm planning on providing some local caching using IndexDB.

My SQL structure uses a large (and variable) number of tables, each table storing an array of data (time sequence and value) for a specific sensor value. Ideally, I would have assumed I'd create a new object store for each of my tables from MySQL. However, it seems that you can only create an object store when the database is opened which is a bit of a pain.

So, I see a number of options -:

  • I could use a single object store and add two indexes - one for the time, and one for the sensor. I'm a little worried that this might have performance issues, but I'm not sure how data is stored under the hood.

  • I could probably detect a new sensor somehow, and open the database with a new version number. This just feels a little wrong to me.

  • I could alternatively use different databases for each sensor, but I've read somewhere that it's not recommended to use multiple databases (although it's unclear why since this is possibly the easiest solution).

I'd welcome any thoughts people have regarding the best structure for this sort of data, that will provide good performance.

cdjackson
  • 137
  • 1
  • 9
  • how big is the data set you plan to cache? – Deni Spasovski Aug 17 '14 at 03:43
  • 1
    To some degree it's up to the user and their system, but as an example, data could be stored at 1 minute intervals (1440 records per day), we might want to cache a weeks data (~10000 records per sensor), and then there could be between 10 and 50+ sensors. So, maybe 500k records - each record just being a time/value pair. – cdjackson Aug 17 '14 at 09:39

1 Answers1

1

If your data sets are independent, example you don't need to combine results from multiple sensors, I suggest you to split them in different tables and/or different databases. Different database option is more convenient for deleting data.

IndexedDB database limit for performance in a single database is for more than 50K data, depending on browser and hardware. I have a couple tests which can measure the speed, just tweak the object size that is inserted and you can test your use case.

If you have less than 10K data per sensor (object store/database) you won't hit big performance issues. One common mistake when inserting batch of data is separate transaction for each insert - this is completely unnecessary, since you can store 10K data with one transaction. If you are working with even larger data set, you can separate the inserting into couple transaction, so you won't block the reading of that database.

Also for every transaction that you do in IndexeDB you need to open a connection, some people use the approach for keeping the one connection alive and reusing it, I prefer the closing and opening a separate connection for each transaction.

Also for faster access, you can store all database info into Local Storage, that way you can track how many databases you have and descriptions for each of them.

Additionally you can take a look at this similar question

Community
  • 1
  • 1
Deni Spasovski
  • 4,004
  • 3
  • 35
  • 52