3

We have a multi-tenant NodeJS application and we are now adding a cache layer for some of our data.

We are aiming to use Redis cache with redis package and we are trying to check our options for supporting multi-tenancy with two main points in mind:

  1. Securing data.
  2. Purging the old data in tenant level.

Our investigation results so far are that we can use separate Redis instances for each tenant - which is not a good solution for us.

Another option we found is to namespace our keys with the "tenant_id:" prefix. This option solves the first point - data is secured now, but we still have the second point to resolve.

Our use case for this point is that one tenant can put a major amount of data which will fill the cache and push out other tenants' data. We want to define our LRU in tenant level, that is - when new data is added to the cache it will evict only the last recently used data of the same tenant.

Any suggestions?

Community
  • 1
  • 1
user2717436
  • 775
  • 1
  • 10
  • 23

3 Answers3

5

While the option to use logical/numbered/shared databases certainly exists, you should refrain from doing so. Not only will your databases share the same config, because Redis is single threaded each tenant will (potentially) interfer with the operation of others and (worst case) block them.

What you should do is use a dedicated Redis process for each tenant. The good news is that Redis is quite lightweight, so there is very little overhead for each instance in terms of resources. The slightly less good news is that this approach will make administration more complex.

If you wish to avoid the admin pain, I recommend that you research the option of using a hosted Redis offering. Depending on your provider, some allow creating multiple dedicated instances at no extra cost.

Disclaimer: I work at Redis Labs, the home of Redis and provider of solutions, including a hosted Redis that behaves like I described above (for that exact purpose btw).

Itamar Haber
  • 47,336
  • 7
  • 91
  • 117
  • How do you add \ remove a database when a new tenant is created or an old one is removed? do you need to add it manually or does Redis Labs have some way to create databases on the fly? – user2717436 Jul 13 '17 at 06:10
  • 1
    I guess you'd need to configure and spin up/down a Redis process for each tenant which is added or removed from your application which, to me, feels cumbersome. You wouldn't spin up a separate Postgres **process** for every new tenant, would you? (A dedicated Postgres database would be fine, though, as that can be easily created programmatically). Maybe Redis isn't the solution to your problem. Or you need to think about setting up a Redis cluster which is beefy enough for all tenants and implement the LRU mechanism on top of it yourself (there should be 3rd party libraries for this). – Tobi Kremer Jul 13 '17 at 07:13
  • RL can provide a REST API to create/delete databases. You can contact the company's support team for more details. – Itamar Haber Jul 13 '17 at 12:21
  • @ItamarHaber I'm just saying that it feels strange to manage a dedicated process for every customer/tenant in a multi-tenant application. I was just using Postgres as an analogy. – Tobi Kremer Jul 13 '17 at 12:54
  • @TobiKremer I can understand how this is counter-intuitive perhaps, but that's how it is... divide and conquer. – Itamar Haber Jul 13 '17 at 13:05
0

Have you investigated using separate Redis logical databases for every tenant?

You can switch the active database with the SELECT command or select the database to use (database-id) when connecting to your Redis instance:

redis://user:password@host:port/database-id

For instance:

redis://user:password@localhost:6379/1

to connect to logical database 1.

Tobi Kremer
  • 618
  • 6
  • 22
  • Can you please elaborate on this option? how it is implemented and what's the LRU configuration options in separate databases? – user2717436 Jul 12 '17 at 10:03
  • AFAIK, the LRU/resource configuration is shared across all databases, so this approach probably won't solve your problem then. At this point, I think separate Redis instances are the only way to go. – Tobi Kremer Jul 12 '17 at 10:07
  • It seems that here - https://stackoverflow.com/a/29232680/2717436 someone is using multiple databases for the same need as ours. – user2717436 Jul 12 '17 at 10:14
  • The databases are completely separated which should make it easy to do things like wipe all keys just for one tenant. But you won't be able to have different config settings (think redis.conf) per database (memory, save strategy, etc.). – Tobi Kremer Jul 12 '17 at 10:21
  • Do you mean that if I separate to multiple databases I will be able to initiate a clear of all data in one of the DB but I will not be able to define automatic policy for each DB? – user2717436 Jul 12 '17 at 11:08
  • Most Redis commands are executed in the currently selected database, only affecting the data therein. For instance, `FLUSHDB` removes everything in the currently selected database. I'm not sure what you mean exactly with automatic policy. Are you talking about Redis' eviction strategies when running out of memory (basically [this](https://redis.io/topics/lru-cache))? Then no: You won't be able to specify dedicated settings for your databases, because the configuration (redis.conf) is shared among all databases. – Tobi Kremer Jul 12 '17 at 11:22
  • Yes, I meant Redis' eviction strategies, and my question is: as you said the configuration is common to all databases in this instance, but does it apply to all databases? that is - when data is added to one database, does Redis consider items in a different database as candidates for eviction? – user2717436 Jul 12 '17 at 14:47
  • 1
    @user2717436 Unfortunately, it seems so: https://github.com/antirez/redis/issues/1674#issuecomment-41159954 – Tobi Kremer Jul 12 '17 at 15:34
  • Thank you! it really seems our only option is going to be multi instances. Can someone elaborate on this option? – user2717436 Jul 12 '17 at 15:43
0

I just stumbled across the same question and found a good solution.. Look in KeyDB (https://keydb.dev/), it's a Redis fork and now supports multithreading. It's fully compatible with Redis, a drop-in replacement for Redis; so you don't have to change any code.

I use it as full text search provider in a multi-tenancy scenario. My data store is MongoDB (has less support for fts); Every tenant has a hash list in KeyDB storing all keywords. Then HSCAN is used to query KeyDB (it's really fast). The results can then be resolved by MongoDB. Since KeyDB is non-blocking and multi-threading there is no need for complex deployment solutions.

And yes, you can use the existing nodejs libs for Redis to access KeyDB.

Stay healthy, best!

Tom
  • 51
  • 3