148

Microservices architecture suggest that each service should handle it's own data. Hence any service (Service A) dependent on data owned by other service (service B) should access such data not by making direct DB calls but through the api provided by the second service (service B).

So what does microservices best practices suggest on checking foreign key constrains.

Example: I am developing a delivery feature (microservice 1) for products and certain products are deliverable to only certain locations as mentioned in the products table accessible to only products micro service (mircoservice 2).

How do I make sure that microservice 1 (i.e delivery feature) does not take an order to a unserviced location. I have this question because delivery feature can not directly access products database, so there is no constraints applicable at DB level when a delivery order is place in to delivery data base (no check is possible to see if a foreign key match exists in products database or table).

6 Answers6

117

It is possible to use a shared database for multiple microservices. You can find the patterns for data management of microservices in this link: http://microservices.io/patterns/data/database-per-service.html. By the way, it is a very useful blog for microservices architecture.

In your case, you prefer to use database per service pattern. This make microservices more autonomous. In this situation, you should duplicate some of your data among multiple microservices. You can share the data with api calls between microservices or you can share it with async messaging. It depends on your infrastructure and frequency of change of the data. If it is not changing often, you should duplicate the data with async events.

In your example, Delivery service can duplicate delivery locations and product information. Product service manage the products and locations. Then the required data is copied to Delivery service's database with async messages (for example you can use rabbit mq or apache kafka). Delivery service does not change the product and location data but it uses the data when it is doing its job. If the part of the product data which is used by Delivery service is changing often, data duplication with async messaging will be very costly. In this case you should make api calls between Product and Delivery service. Delivery service asks Product service to check whether a product is deliverable to a specific location or not. Delivery service asks Products service with an identifier (name, id etc.) of a product and location. These identifiers can be taken from end user or it is shared between microservices. Because the databases of microservices are different here, we cannot define foreign keys between the data of these microservices.

Api calls maybe easier to implement but network cost is higher in this option. Also your services are less autonomous when you are doing api calls. Because, in your example when Product service is down, Delivery service cannot do its job. If you duplicate the data with async messaging, the required data to make delivery is located in the database of Delivery microservice. When Product service is not working you will be able to make delivery.

Ali Sağlam
  • 1,410
  • 1
  • 10
  • 15
  • 4
    Great answer. I use API calls, but it also needs sorting and pagination of the data from another service. Do you know the best approach for that case ? – tranceholic Dec 20 '17 at 10:20
  • 8
    You should add the parameters related to paging and sorting to your api. Then the responsibility of getting the right page with the right order will be taken by the consumers of the api. There are some technologies used to define an api like GraphQL. As far as i know, those technologies already has sorting and pagination features. If you are not using this kind of technology, you can simply get the parameters from your client and use them to return your data sorted with pages. – Ali Sağlam Dec 21 '17 at 11:14
  • 2
    Great answer indeed! – T.S. Apr 13 '18 at 23:53
  • 5
    But, do you keep the foreign key though? Ex: Every blog post will have many comments. Monolith will have comments table with foreign key to blog post. However in microservice, we will have two services. Service 1 : Post Microservie with these table fields (PostID, Name, Content) Service 2 : Comments Microservie with these table fields (CommentID, PostID, Cpmment) The question is, Do we need "PostID" in service 2 (Comments Microservice) ? I guess the answer is yes, as we need to know which comment belongs to which post. Is my understanding correct? – rakesh mehra Jul 15 '20 at 11:44
  • 3
    How to divide a system into microservices is totally a different story, but if you decided to create 2 microservices like post and comment you need post identifier on comments microservice as each comment belongs to a post. However, it doesn't mean you need to define FK between those tables. FK is only a constraint in RDBMS world which helps ensure data integrity and consistency. If you keep these microservice's data on separate schemas you can't define FK or even you may keep one's data on a nosql db (which would make sense for comments microservice) where FK is not applicable. – Ali Sağlam Jul 16 '20 at 12:17
  • 1
    post/comment example, when a post is deleted, then in the post service, it need do an api call to the comment service to delete the comments. The dependency at db level surfaces to the code level doing api calls. But of course, now we have two databases. – user3453552 Sep 27 '21 at 23:24
  • 1
    @user3453552 Posts service could just produce a `PostDeletedEvent` that all interested parties (e.g. Comments service) will listen. This decouples Posts service from Comments service. But ofc Comments service will be coupled to Posts service in some way, as comments can't exist without a post. – Ruslan Stelmachenko Dec 03 '21 at 04:38
  • 1
    Superb answer. It helped me a lot. – Ketan Patil Jan 13 '22 at 14:05
  • what about operations like insert, for example how to validate foreign keys of an insert that consist of lets say 10 Fk's and all those keys are in another services . how to check if all those Ids are valid for every insert operation of a service ? – sasan Feb 21 '22 at 14:57
  • @sasan If you need to check if dependent objects on every insert operation, may be you should think of merging those two services, as it seems they cannot operate independently. Alternatively, you can copy FK object to the other service in an async way (Data duplication). There are lots of patterns and anti-patterns are now around and I think this question is more related to how you should decomponse microservices and how you should design the interfaces between them. I believe this is the hardest part of Microservices architecture. – Ali Sağlam Feb 21 '22 at 16:59
  • ^^ TL;DR Don't design such a system that a microservice is heavily dependent to another one like it needs to check references every time. – Ali Sağlam Feb 21 '22 at 17:00
  • Having redundant data for all common entities in all databases? I think it make a chaos. – FLICKER Feb 22 '23 at 02:55
35

When distributing your code to achieve reduced coupling, you want to avoid resource sharing, and data is a resource you want to to avoid sharing.

Another point is that only one component in your system owns the data (for state changing operations), other components can READ but NOT WRITE, they can have copies of the data or you can share a view model they can use to get the latest state of an object.

Introducing referential integrity will reintroduce coupling, instead you want to use something like Guids for your primary keys, they will be created by the creator of the object, the rest is all about managing eventual consistency.

Take a look at Udi Dahan's talk in NDC Oslo for a more details

Hope this helps

catch22
  • 1,564
  • 1
  • 18
  • 41
Sean Farmar
  • 2,254
  • 13
  • 10
12

first solution: API Composition

 Implement a query by defining an API Composer, which invoking the
 services that own the data and performs an in-memory join of the
 results

enter image description here

second solution: CQRS

Define a view database, which is a read-only replica that is designed to support that 
query. The application keeps the replica up to data by subscribing to Domain events 
published by the service that own the data.

enter image description here

Ali_Hr
  • 4,017
  • 3
  • 27
  • 34
4

A 2020 update to this answer is to use a Change Data Capture tool like Debezium. Debezium will monitor your database tables for changes and stream them to Kafka/Pulsar (other pipes) and your subscribers can then capture the changes and synchronize them.

user521990
  • 669
  • 1
  • 5
  • 21
4

...How do I make sure that microservice 1 (i.e delivery feature) does not take an order to a unserviced location...

You don't do it online, but in a deferred way.

Your service #1 receives the order, perform all validations it can do by itself, and saves it. A deferred service, processes the order and validates the other aspects of it later on. It may come back as rejected, once the location is found to be non-serviceable. Your service will need to gracefully inform that to the user, and maybe even cancel the order.

The Impaler
  • 45,731
  • 9
  • 39
  • 76
-3

Foreign key comes under when you're having more than two tables, you want make any relation between tables, then at that time a foreign key is used.

For Example:

Table-1: Student Table has these two fields, SID And SName [SID Means Student_Id],....

Table-2: Management Table has these two fields, SID And StaffId.

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
RAVULA BIKSHAPATHI
  • 1
  • 2