Grouping data together in Microservices

Question

Lets suppose I have two services in a mircoservices architecture which are:-

• OrderService
• CustomerService

Each service has its own database, my frontend makes a request to /api/order/25 now along with the order I want to return the CustomerName as well but CustomerName is in a different database. So whats the best and most microservices based approach for fetching the CustomerName along with the Order data?

Answer 1

In this case, I suggest creating an aggregator service or backends-for-frontend. This service will fetch data from both the services and aggregates it for your frontend(or for your client).

In this way, your order service doesn't need to depend on customer service for just getting a name or your client doesn't need to make another call for fetching data and do aggregation.

With BFF,

• You can add an API tailored to the needs of each client, removing a lot of the bloat caused by keeping it all in one place.
• Frontend requirements will be separated from the backend concerns. This is easier for maintenance.
• The client application will know less about your APIs’ structure, which will make it more resilient to changes in those APIs.

However, all these microservices patterns came with some tradeoffs and no exceptions for BFFs. so Always keep in mind that,

• BFF is a translation/aggregation layer between the client and the services. When data is returned from a service API, the purpose of it is to aggregate and transform it into the data type specified by the client application.
• Avoid over-dependence on BFF and don't add application logics in this layer. As I said in the previous step it's just a translator.
• Implement a resilient design and timeout since this aggregator is calling other services and getting data. If one or more service calls take too long, it should timeout and return a partial set of data. Consider how your application will handle this scenario
• Monitoring of your aggregator and it's child service calls. Implement distributed tracing using correlation IDs to track each call.

Answer 2

You have multiple options here. For example:

1. Duplicate some Customer data in the OrderService Database. You could for example save a small subset of the entity as a separate Table in the Order service db like: CustomerId and CustomerName. This works very well for use cases where you have a lot of traffic on the: /api/order/id endpoint but the number of Customers is very small. This means that duplicating some of Customers data in your OrderService db would not be very expensive. This option works quite well if you have a Messaging Queue for communication between micro-service which publishes event messages when the Customer data changes in CustomerService. This way you can update the Customer data in OrderService db and be up to date with the source of truth of the data. If you do not have a Messaging Queue for communication this option would not be so great as you would need to periodically check for changes in the CustomerService. This will create additional load to the CustomerService and the Customer data in OrderService db would inconsistent for some period. Good thing about this option is even if you have a lot of calls to your "/api/order/id" the CustomerService will not be affected at all by this load as the data is already in the OrderService db.

2. Call the CustomerService from OrderService when you need that data. In this option every time you need the CustomerName and other CustomerService data you would need to call the CustomerService over some api. For example that would be when someone calls your "/api/order/id" endpoint you would call the CustomerService api from your OrderService. This option is very good if you do not have a lot of calls to this api and generally the load on the CustomerService is not big. This way this additional calls would not be very problematic. It becomes a problem if you have a lot of calls to"/api/order/id" and the load is delegated to the CustomerService as well.

3. Create a 3'rd micro-service which will aggregate data from both the CustomerService and OrderService. In this option you would have another read-only micro-service with its own database which will have data from both micro-service in its own database. This option can be useful if you have a system which has to handle a lot of load in particular has a lot of potential for lot of load to this particular endpoint: "/api/order/id" or other endpoints which use an Aggregations of multiple entities from multiple micro-service. If you want to separate this and scale this independently you can create another micro-service like customer-order-micro-service and duplicate data from CustomerService and OrderService. Similar as in option 1. you would just have to duplicate particular fields/properties of the Entities which you need. Using this option you could even have some kind of De-Normalized data structure where you could store the entities in one table/collection(if you need it :)). You can even use pick a data storage technology which fits your queries better like Elastic Search if you have some full text search requirements and etc. This special service can be used to fit your needs as you need it. Some examples are also in the direction Query/Read part of CQRS and so on. There are a lot of options what you can do here. This is an option for a specific use case so you need to carefully review your business requirements and adjust it to your needs.

Conclusion

Usually in most of the cases you will be fine with option 1 or option 2. But it is good to know that you can also decide to go with the option 3 and have a special read-only/query only service which can server your special needs. Keep in mind that the way of communication between services has also an impact on your options. You can read more about micro-service to micro-service communication here.