4

I'm diving into Service Fabric (from the Cloud Services world) and am hitting a few speed bumps with how ReliableQueues work.

Let's say I have 2 stateful services StatefulService1 and StatefulService2.

If I need to have StatefulService1 send a message in a queue that StatefulService2 will pick up and read am I able to use ReliableQueues or are ReliableQueues isolated within the service they are created in?

If that is the case then what is the purpose of having ReliableQueues? The usual pattern behind them is for another process to act on the messages. I understand why isolating a Dictionary to a service would make sense, but not a queue...

Is my best option to rely on a traditional approach to send this message such as a Storage Queue or does ServiceFabric offer a solution for passing message queues between services?

UPDATE

Just want to clarify that I did attempt to dequeue a message created in StatefulService1 from within StatefulService2 and it came up empty. Dequeing from within StatefulService1 worked fine as expected.

INNVTV
  • 3,155
  • 7
  • 37
  • 71

4 Answers4

4

Reliable Collections are in memory data structures that are not intended for inter-service communications. If you would like to establish a communication channel between StatefulService1 and StatefulService2, you have the following options:

  1. Use Communication Listeners. You can have custom listeners for the protocol of your choice, including HTTP, WCF or your custom protocol. You can read more about it in this section. For example, StatefulService2 can open up an HTTP endpoint that StatefulService1 can POST/GET to.

  2. Use an external queuing system, like Servicebus, EventHub or Kafka, where StatefulService1 can post events to. StatefulService2 can be consumer service that consumes events from the queue and processes it.

Sujay S Kumar
  • 621
  • 1
  • 5
  • 10
3

Reliable Collections (queue and dictionary) are not intended for communication. With queues, it's a 2PC, so only one process can access it at any point in time. Note that when you use stateful services with partitions, to access the data both service instances have to be on the same partition. Different partitions cannot access the same data.

Relying on either traditional methods or implementing your own communication listener is the way to go. With the traditional way - keep in mind that you'll need to decide if you want to partition your queues just like your services are or not.

Sean Feldman
  • 23,443
  • 7
  • 55
  • 80
  • Thanks Sean. Can you help me understand how a ReliableQueue should be used then? Like I said ReliableDictionary makes sense as it replaces a cache. But if you have a Service submitting messages into a queue in what scenario would you have that same service reading from a queue? To me a queue is a way to have some other process handle tasks through decoupling, Thanks! – INNVTV Mar 05 '17 at 22:11
  • A reliable queue is great to enqueue work items. It's not a communication mechanism between services, but an inmemory data structure for a single process. For example, get work items and enqueue those so that another thread (under that service) can access it for actual processing. – Sean Feldman Mar 05 '17 at 22:30
  • Thanks. That makes sense. So rather than (for example) having a WorkerRole that runs tasks for many services I can now have the service itself run these tasks in a separate thread. – INNVTV Mar 05 '17 at 23:12
  • Remember, communication between services would still require something. – Sean Feldman Mar 06 '17 at 00:54
1

I don't see why a service can't host a reliable collection/queue, and other services can access it via one of three transports: Remoting, WCF and HTTP.
Obviously, the reliable service will have to expose the collection/queue via an API or implement an IService interface

https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-connect-and-communicate-with-services

I Stand With Russia
  • 6,254
  • 8
  • 39
  • 67
0

You have to add a fault-handling retry pattern to your calling code, see https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-communication, in this case you don't need a queue to hold data for your in between service calls.

Robert
  • 176
  • 7