1

I am trying to develop microservices for online shop using CQRS, DDD, and Event sourcing concept. I looked to AWS Kinesis as event stream. I think it would be good for choreographed microservices. I have 2 services, service for customer data and service for ordering system. I want to see total number of unpaid orders and the total amount of orders for each customer. So, I should send orderCreated event and orderPaid event to the service for customer data and recalculate the total unpaid orders and total amount of orders for related customer.

Could I put the ordering system events to AWS Kinesis and listen it in command-side service for customer? Should I persist the events (orderCreated and orderPaid event) from AWS Kinesis to database in customer command-side service? Or is it ok to just update the customer query-side service only? Should I use AWS Lambda as event processor? Could you give me some best practices for this model?

Thanks in advance.

Benedict
  • 61
  • 1
  • 6

1 Answers1

1

I looked to AWS Kinesis as event stream. I think it would be good for choreographed microservices.

I don't think that's the use case that Kinesis is designed for; see this overview by Aditya Krishnan. Or this previous question on stack overflow.

Should I persist the events (orderCreated and orderPaid event) from AWS Kinesis to database in customer command-side service?

From my point of view, the issue is this: you really don't want events to leak out to subscribers and then not appear in the book of record. So the usual ordering is to put the events into the durable store, and only after you get the acknowledgement of the write (which is a proxy for "acknowledgment that we've reached our minimum durability guarantee"), then you start sharing the events out.

So most designs reverse the order you propose - durable store (database) first, publish second. But you are losing latency; subscribers can't see the event until the store finishes. Depending on your design, you might be able to make up some of that with batch reads.

We have tried the SQS and SNS. But, the performance is not good enough. It takes about 5 seconds to publish and consume the events.

Hmm, given what I see in the recommendations for Kinesis, it doesn't look like you are going to beat that by more than an order of magnitude; they seem to be recommending full pipes rather than fast ones.

VoiceOfUnreason
  • 52,766
  • 5
  • 49
  • 91
  • Could you give suggestions what the tools to handle this issue? Actually, we insert the event records in database. But, we should publish some events to another service. We are looking for the tools to consume these published events. We have tried the SQS and SNS. But, the performance is not good enough. It takes about 5 seconds to publish and consume the events. So, we want to try another way to solve this issue. – Benedict Aug 03 '17 at 03:43
  • I disagree. Event streams are a great use case for kinesis. It just requires a bit more work that SQS and SNS. – Mike May 15 '18 at 14:57
  • @Benedict When you say you 5 seconds end to end time in SQS, were you using short poling or long polling? What kind of throughput you have for the queue? If you use short polling, and poll more aggressive, you should be able to achieve much lower end to end time. – yoroto Nov 13 '18 at 23:04