16

I have a few different RESTful services that are hosted on different servers which use different DBs. I have a few RESTful services which call multiple such services above in what is supposed to be a transactional unit. We end up with data consistency issues if any of these RESTful services fail. Is there a neat architectural way of orchestrating a rollback? Or is having transaction managers the way to go?

As a simplistic example, RESTful service 1 has a POST request which reduces item count of thingamajig by 1. RESTful service 2 POSTs a payment. If service 2 fails, how can we cleanly implement a rollback on service 1, without having a new RESTful refund service (it is ok if this has to be the way to go). I am looking for an architectural answer to above issue, which is in keeping with REST principles.

cvam
  • 163
  • 1
  • 1
  • 4
  • Can't you use DDD and eventual consistency? That way you won't need rollback on fail, just retry. I think it would be easier than implementing multi phase commit on top of REST services. – inf3rno Oct 12 '15 at 11:00
  • Similar question: http://stackoverflow.com/a/3837376/607033 – inf3rno Oct 12 '15 at 11:01

3 Answers3

15

Your answer: https://stackoverflow.com/a/1390393/607033 You cannot use transactions because by REST the client maintains the client state and the server maintains the resource state. So if you want the resource state to be maintained by the client then it is not REST, because it would violate the stateless constraint. Violating the stateless constraint usually causes bad scalability. In this case it will cause bad horizontal scalability because you have to sync ongoing transactions between the instances. So please, don't try to build multi-phase commits on top of REST services.

Possible solutions:

  • You can stick with immediate consistency and use only a single webservice instead of two. By resources like database, filesystem, etc. the multi phase commit is a necessity. When you break up a bigger REST service and move the usage of these resources into multiple smaller REST services, then problems can occur if you do this splitting wrongly. This is because one of the REST services will require a resource, which it does not have access to, so it has to use another REST service to access that resource. This will force the multi phase commit code to move to a higher abstraction level, to the level of REST services. You can fix this by merging these 2 REST services and move the code to the lower abstraction level where it belongs.
  • Another workaround to use REST with eventual consistency so you can respond with 202 accepted immediately and you can process the accepted request later. If you choose this solution then you must be aware by developing your application that the REST services are not always in sync. Ofc. this approach works only by inner REST services by which you are sure that the client retry if a REST service is not available, so if you write and run the client code.
  • Another, probably the ugliest workaround to store every transaction as a resource, so you could POST commits and rollbacks. I think this possible solution is not viable, because it would violate the uniform interface constraint. We would use POST /transactions/ {resource: "/forums/12/messages/45", method: "PUT", data: "..."} and POST /transactions/1/commit instead of for example PUT /forums/12/messages/45
Community
  • 1
  • 1
inf3rno
  • 24,976
  • 11
  • 115
  • 197
  • Some extension to the answer after 7 years: I would use event sourcing and a compensation mechanism if we need to rollback instead of reverting everything. The REST services can be called based on the stored events and retried countless time before they give up and we need to compensate (usually manually if it is rare and it is hard to automate). Having a refund service is a great example of a compensational mechanism, so I think it is the way to go. As of the last point it is possible to have transaction like resources when it takes multiple steps to solve something e.g. payment,delivery,etc. – inf3rno Sep 01 '22 at 15:01
6

Distributed transactions are complex and require each participating system to support a notion of rollback. In the case of your services, they would each have to support some form of rollback. Co-ordinating something like this in a distributed system may not be practical or advisable in a synchronous way. In a situation like this, you would want to asynchronously roll back and the system would eventually reach consistency at a certain point in the future. There are obviously many other details (timeouts, error handling, retries, etc.).

For more details on eventual consistency, check out the wikipedia entry here.

leeor
  • 17,041
  • 6
  • 34
  • 60
  • although other commenters have proposed something similar, combining 2 different services into one (tht might belong to 2 teams, serving different verticals/customers) is not feasible. since this answer is more pertinent to soving this issue in a multi team corporate env, I am accepting this. – cvam Oct 15 '15 at 02:35
0

Basically the problem is that you need to use transactions in an environment (HTTP) that by default is not transactional - in DB sense (because in HTTP a transaction is a successful request - response cycle).

The content of @leeor response is fully correct, what I'd like to add is how I'd solve the problem from the design site.

So you need a single endpoint, may be /transactions. Via POST method you add a new transaction (with all the necessary details) that is immutable - after creation you can only ask for it's data/status via GET method. What can update the transaction status/data is the server itself only.

Under the hood (during transaction creation) a snapshot (that could be reversed later on) for each resource taking part in the transaction should be created. Then execution for all the operations should begin and in case of any fail all the snapshots should be reversed. You do not mention any technologies so it's really hard to advise something reasonable. What you'd need for sure is comprehensive logging.

Opal
  • 81,889
  • 28
  • 189
  • 210