-1

I'm trying to build an Event Driven Microservices Architecture, as I understand It's recommended to build my services without a DB, and instead to use the Event Store technique which is based on Event Driven Microservices Architecture.

My question is, if my services would be small and totally independent from each other, including not have a dedicated DB for each service, should my Event Store act as one single unit \ "service" which holds other services events'?

If yes, one of the Event store components is a message bus (like apache Kafka), in order that services could consume and publish events, does it mean that the Event Store domain is virtual? (because whole its components including Kafka doesn't being packaged as a single unit).

Moshe Arad
  • 3,587
  • 4
  • 18
  • 33

2 Answers2

1

I would not recommend building an app that has to persist data without any permanent store. Even though it is possible to store an event queue forever it is not very good for random data access. Imagine your app needs to access some user information that is stored in the middle of the queue. Since you dont have the event ID you'll have to reprocess the queue in order to find that information which will be very slow.

The event queue is useful to decouple service dependencies but it is not a good permanent data storage. Typically you'll want to process the queue with service dependent consumers that transform and move the data into a format and storage as useful for the service.

Also see this answer

Community
  • 1
  • 1
Oswin Noetzelmann
  • 9,166
  • 1
  • 33
  • 46
0

An event store can be nothing more than a journal of events that is replayed, in full, to regenerate an service's original state. If you use a compacted topic in Kafka, you can minimise the restore time (a compacted topic just drops old events for the same key). This is fine for runtime state.

There are a number of options for facilitating queries. If you don't mind getting into the whole KStreams thing, the simplest is to materialise a queryable view in a KTable or State Store. This is a database (it's using RocksDB behind the scenes) constructed inside your service. It acts as a disk-backed cache over the data in the backing log. This has the useful property that the backing stream can be shared by many services, but the materialized view is owned entirely by each service.

More generally, a good approach is to do the simplest thing that will work, then evolve it. Try to keep services stateless and event driven. Pull in KTables or states stores if your requirements necessitate stateful elements. If your data requirements grow, look to branch out into an independent database. If you started with a kafka-backed store you can typically migrate the data relatively easily with the Connect api (although your logic may be affected).

One trick worth noting for this type of implementation is to avoid synthesising request-response channels between services. Instead follow and Event Driven Architecture where you build up a shared narrative of events. Martin Fowler has a good write up on this from a while back. He calls it Event Collaboration.

Ben Stopford
  • 156
  • 2