Can a microservice have multiple deployables? Is CQRS a valid use-case for having 2 microservices/deployables sharing a database schema?

Question

Example use case: There is one microservice which is holding terabytes of data in few tables which all belong to the same business domain.

• The import of data happens via AMQP messages in an event-driven manner (in CQRS terms - commands). The processing of a message with imported data might take up to some minutes. Losing such a message/command is not allowed.
• The imported data needs to be made accessible to other microservices of other bounded contexts via AMQP messaging in a way that a request/response message cycle takes maximally 1 second (in CQRS terms - queries). Losing a query message is allowed.

Initial considerations: The scalability, elasticity, performance, fault-tolerance and maybe some other -ility requirements for commands and queries are different. It also seems to be also tempting to isolate the query processing from command processing in different deployables, in order to address the -ility requirements for commands and queries processing in a best possible manner, by

• Avoiding cases, where query processing gets blocked by the long processing of commands
• Avoiding out of memory errors when commands and queries are processed at the same time
• In some tech stacks, such as Spring AMQP, the configuration for message delivery guarantees with lower performance vs. maximal performance with no message delivery guarantees are done centrally for the whole application, e.g. message acknowledgments, concurrency and etc. This might negatively impact query message processing performance

But then we might get into discussion of the following questions:

• Do we consider 2 deployables (1 for slow-command command, 1 for real-time query processing) as 2 microservices?
• Sharing a database schema between that 2 microservices/deployables is an anti-pattern and should not be done? vs. But we get benefits in terms of decoupling of the -ilities for commands and queries?

What are your thoughts and arguments on that?

Answer 1

Personally, I think microservices was a bad choice for renaming doing service orientation right. The "micro" in the name drives people to build small services while adhering to all the principles (like autonomy) resulting in what I like to call nanoservices - service's whose utility is smaller than the overhead it incur. You can see that companies, like Uber, that went all in and got a big ball of distributed mud are taking a step back to rethink service boundaries vs. using multiple executables.

I think about these as aspects of a single service. So the service is about handling this data and it has an ingestion aspect and a query aspect and they need to scale and develop independently. As long as you guard the boundary of what's inside and and what's outside of the service, the coupling <-> freedom tradeoff within the service can be controlled

So yes, microservices "by the book" will require you to treat each of this deployable units as a single service with its own database etc. but you don't need cargo-cult decisioning to drive your architecture, you need to make deliberate choices that will serve your business and technical needs

Answer 2

My understanding of a microservice is that it is essentially self-contained: can be deployed independently, and is loosely-coupled to other microservice - but ideally loosely-coupled to anything. Further info here.

Having a microservice consuming a database/schema/whatever directly would be ok if the database was an internal and exclusive part of the microservice. But in your case, it's a shared resource. So you have something that is both possibly tightly-coupled and shared.

One thing to always ask yourself is "what problem am I trying to solve". Microservices are a powerful approach, but not a silver bullet. If you have something which attracts a set of strong constraints then these can sometimes challenge the broader architecture - so microservices might be good for 80% of the solution but not 100%. Strong constraints that might force you to make specific architectural decisions include non-trivial NFR's, system constraints (e.g. brittle hard-to-change legacy systems), etc.

So, what should you do? Some ideas:

1. If logic / analysis has brought you to your current option, and the solution seems sound, then maybe it's the right one. If it doesn't fit the microservices architecture style then that is not "wrong" in of itself - your solution just isn't microservices.
2. Accept that your two "microservices" aren't really microservices, but logical parts of some slightly larger specific system. Even so, you can still apply sensible architectural rules to it: appropriate application of SOLID, etc.
3. Keeping the query and import functionality logically separate sounds wise, you may even be able to architect it in such a way that they can be physically separate as well.
4. If you develop this particular system, which is not microservices like the rest of your architecture, then you'll need to pay appropriate attention to stuff like documentation, architecture, communication to developers, etc, so that everyone is clear why this part of the solution is different.
5. Look at the technology stack you are using and see what specific tools it has for dealing with your problem. Caching might be an area to explore.
6. Related to caching, have a look at the "Write-Behind" and "Write Through" design patterns (and others). Basically there are various patterns for dealing with caches and master records, that give precedence to different drivers. You just need to choose the one that matches your need. This might be a useful starter (which I found through @qwr's answer).