2

I'm building a product which requires creation of a new db dynamically for each project as a new team will create a new project depending on need. See image below. The backend is implemented with Node.js, Express.js, TypeScript and ArangoDB is the database used.

This is a product for enterprise server installation as well as SaaS.

How can I achieve the required architecture with mentioned technologies? Thanks for your ideas and examples, references. enter image description here

kashyap
  • 133
  • 1
  • 10
  • Hi, can you elaborate on reason why you're planning to create new database per project? It has massive cons like 1. you'll loose access to out of box functionality to search/filter typical project aspects like tasks across all projects 2. creating/upgrading collections as well as using any graph, search, foxx features gonna turn into deploy/upgrade nightmare because you gonna have to do it on every database... – sevcik.tk Oct 23 '21 at 12:43
  • Hi @sevcik.tk, Your points are quite valid. My thought is that every project's assets must be kept separate to avoid mixing of documents which might run into hundreds of thousands over a period. I'm not sure if my thought process is right and therefore, the question. I'll be happy to learn if there's architecture advice to deal with such a scenario. – kashyap Oct 23 '21 at 20:04
  • Looking at multi-tenancy architectures, my thought process makes sense and should be possible. I would like to know how to achieve it. – kashyap Oct 23 '21 at 20:47

1 Answers1

2

ArangoDB JavaScript Driver folowing Modern JavaScript/TypeScript with async/await: you can manage a dynamic collection of new Database() instead of one const db. It can be a map with a key of the DB owner at each team and a value of Database object. Still joining data between two DB connections is not easy.

Quicksql is a SQL query product that can be used for specific datastore queries or multiple datastores correlated queries. It supports relational databases, non-relational databases, and even datastore which does not support SQL (such as Elasticsearch, Druid). In addition, a SQL query can join or union data from multiple datastores in Quicksql. For example, you can perform a unified SQL query on one situation that a part of data is stored on Elasticsearch, but the other part of data is stored on Hive. The most important is that QSQL is not dependent on any intermediate compute engine, users only need to focus on data and unified SQL grammar to finish statistics and analysis.

Yet some custom ArangoDB support is needed to use it with ArangoDB Java Driver.

Tomer Bar-Shlomo
  • 425
  • 5
  • 8
  • Hello Tomer, Thanks for your response. I've seen the drivers. Java is definitely not an option because application is in TypeScript. I was thinking along the same lines like you. I've read somewhere in ArangoDB documentation that we can't keep multiple connections to different databases at the same time. This in general limits the application of the database without a proper driver to support this functionality. I know there are ways in which the desired architecture can be achieved. I would like to listen from ArangoDB team about what their proposition is. Thanks. – kashyap Oct 25 '21 at 10:47
  • The application requires that all teams have simultaneous connection to their database/workspace. So all of them have to be available without swapping connection to any database. – kashyap Oct 25 '21 at 10:50