5

I would like to know which pattern is recommended to work with counter representing the number of message processed, with an application that should be stateless.

For example, in an architecture where an application is deployed on several servers, a database is used to store the persistent information (session, etc...). However, such information are not exposed to concurrent updates like a message counter would be. In a mono-instance application we could use singleton, but this is not the case here.

What would be your suggestion to implement a such counter ? Is using a counter a bad design ?

Anthony Corrado
  • 61
  • 4
  • That would depend a lot on what you are going to use the counter for. What is the use case? – K Erlandsson Jun 30 '15 at 20:01
  • For instance, we could receive a lot of transactions (quantity is known) for which the treatment is dispatched on several servers. At the end of the treatment of all the transactions, a report should be generated. But, how to know that last transaction has been processed ? – Anthony Corrado Jul 01 '15 at 17:35
  • Count transactions on each server independently and add those numbers up when you need a total. Each server responsible for it's own synchronization. – WW. Jul 03 '15 at 10:48

3 Answers3

1

I may not be directly answering to your question, but I can give you a reference of a counter service available, which is multi threaded , multi noded, scalable and at the same time considers availability scenarios. Check the jgroups counter service http://jgroups.org/manual/index.html#CounterService.

This can guide you to a set of problems in the scenario of a distributed counter and also act as a live working reference.

arunvg
  • 1,209
  • 1
  • 20
  • 31
  • I do not think it solves the problem. There is still only one counter coordinator, that does actual incrementing. This can be only one machine at a time and @AnthonyCorrado wants to avoid this. `In a nutshell, in a cluster the current coordinator maintains a hashmap of named counters. Members send requests (increment, decrement etc) to it, and the coordinator atomically applies the requests and sends back responses.` – Vladislav Rastrusny Jul 02 '15 at 15:18
  • If the counter needs absolute consistency, this should be a reference model for (in memory), or we can go for an database approach ( which induces more latency). But for sure, if we need only an eventually consistent counter we would have more options for design. – arunvg Jul 03 '15 at 04:00
  • 1
    What type of counter is needed only question author knows ;) We can only suggest solutions. – Vladislav Rastrusny Jul 08 '15 at 14:38
0

I do not think there is some pattern to do this very abstracted task. Also:

In a mono-instance application we could use singleton, but this is not the case here.

Using a singleton DOES NOT guarantee data safety in a multithreaded system. You need some synchronisation primitives. The very simple is a critical section.

If all of your applications should update some counter, it looks reasonable to refactor this... "counter" management into a microservice and use critical section inside microservice to update resource. In this case in each given moment in time you are guaranteed to have only one thread updating counter.

Vladislav Rastrusny
  • 29,378
  • 23
  • 95
  • 156
  • I think this could be solution to this problem, but imagine we have billions of transactions, the microservice will receive requests from dozens of servers and all these requests will be condensed on one server for which horizontal scalability is not possible. – Anthony Corrado Jul 01 '15 at 17:48
  • @AnthonyCorrado You cannot make a thread-safe monotonically incrementing counter without synchronisation. What is the real task behind this? – Vladislav Rastrusny Jul 02 '15 at 08:00
  • for instance, we could receive a lot of transactions (quantity is known) for which the treatment is dispatched on several servers. At the end of the treatment of all the transactions, a report should be generated. But, how to know that last transaction has been processed ? – Anthony Corrado Jul 02 '15 at 13:41
  • @AnthonyCorrado Could you assign a GUID to each transaction and store GUID list somewhere (it should be stored in DB at least)? You will only need to count GUIDs to verify if the last transaction was processed or not. – Vladislav Rastrusny Jul 02 '15 at 13:50
  • It could be yes. But, what happens if a new transaction arrives between the INSERT and the COUNT ? We can have a situation where several messages are considered as LAST, no ? Is there a method to "synchronize" INSERT and SELECT COUNT into a DB ? – Anthony Corrado Jul 02 '15 at 15:03
  • @AnthonyCorrado If you have a distributed system, in order to detect the end of treatment dispatch, you just periodically poll database for COUNT() until it reaches your desired value. Until then, treatment should be considered still dispatching or whatever state you call it. – Vladislav Rastrusny Jul 08 '15 at 14:35
0

Possible solutions with a database:

  • create a stored procedure that will update the counter and return the new value (UPDATE ... RETURNING ... INTO) in Oracle for example
  • $inc in mongodb for example

counter will be stored in the database and will be available on each application instances that will be connected to this database

Anthony Corrado
  • 61
  • 4