1

Say I created a stateful service fabric service with 5 partitions on a 5 node cluster. I see that each node gets 1 partition per node. When I debug the service in VS, I notice that service fabric is creating exactly 5 instances of the stateful service [essentially 1 static instance per partition] across all 5 partitions. No mater how many calls the clients make, there are only 5 instances of this class to serve requests from. Are the following statements true?

  • Any class level member variables in the stateful service are essentially static [since it resolves to a singleton instance on that partition] and therefore require "lock" semantics when updating?

  • Since the client always resolves to a the "same" instance of the class all the time for a given partition, the client can re-use the "proxy" instance?

  • How does this "singleton" model of stateful service creation affect performance and scalability when lot of clients call the same service instance hundreds of times a second. [under heavy load] I understand that there is a setting when configuring the V2 remoting listener where we can specify "MaxConcurrentCalls" via "FabricTransportRemotingListenerSettings".
teeboy
  • 408
  • 3
  • 13
  • Hi @teeboy. Can you clarify how many replicas do you have per partition? Is it one replica per partition? – Oleg Karasik Jun 19 '18 at 12:00
  • I just realized that I changed my local cluster to 1 Node. Does service fabric ignore "replica" counts for 1 node clusters? But, I do have replica count as 1 in local.1Node.xml. – teeboy Jun 19 '18 at 12:04
  • Service Fabric doesn't ignore 'replica count' on 1 node cluster - there is a constraint to have no more than one 'replica' from single partition one a node. Getting back to your question - Service Fabric creates an instance of StatefulService class for each replica. This instance will be alive until replica is shutdown. To get more insight I would recommend you to check [service lifecycle](https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-lifecycle). – Oleg Karasik Jun 19 '18 at 12:12
  • Thanks for the link. I just re-read it. It does not mention anywhere if fabric run-time creates multiple instances for each client call or a singleton instance is created per partition. – teeboy Jun 19 '18 at 12:26
  • Service Fabric **doesn't** create a new instance of StatefuleService class for each client request rather than that it creates one instance per replica. To try it you should create 5 nodes cluster and specify **TargetReplicaSetSize** and **MinReplicaSetSize** to 5 (then you would have 5x5=25 replicas in total). Also please note that by default secondary replicas doesn't do any work i.e. listening to endpoints (you should explicitly specify that) and by default all client classes would always contact _primary_ replica. – Oleg Karasik Jun 19 '18 at 12:38
  • Ok, that confirms that the client always hits the same instance for a given partition. So, it's safe to assume that *all* class level variables are essentially static? [even though they not marked as such]. – teeboy Jun 19 '18 at 13:33
  • as I mentioned there many things done by default. If you won't define listeners to listen on secondary replicas like [here](https://stackoverflow.com/a/38152967/9497714) then yes you will always hit primary replica. The thing here is that primary replica can be moved to a different node or demoted by some other reason and another replica will become primary and will start handle requests. See promotion and demotion [here](https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-lifecycle#stateful-service-primary-swaps). So it itsn't safe to assume that. – Oleg Karasik Jun 19 '18 at 16:09
  • Just to clarify, my question is specifically related to the object instances being created by service fabric runtime. When I debug the app in Visual studio and do "this.GetHashCode()" on a stateful service, it always returns the same hashcode for a particular partition. This was what I was referring to. I am not talking about "replicas". replicas will probably have their own object instance – teeboy Jun 20 '18 at 15:29
  • I think we have a misunderstanding here. What I am talking about is that there is no class instance created for partition. The class instances are created only for replicas. Each partition has at least one replica so if you create a partition with only one replica then yes all request are pointed to that particular class instance. But this is true only while replica isn't recreated or moved. Then class instance would be terminated and a different one will be created. – Oleg Karasik Jun 20 '18 at 19:44
  • Yes, I am talking about the primary replica only. For ex: in a server app like asp.net, every request to a webapi controller creates a new instance of the controller class. I just wanted to see if the same holds true for stateful service class instance in service fabric. Looks like each partition has 1 instance of the stateful service [only talking about primary replica here]. And this same instance [essentially a singleton instance] will service all requests. – teeboy Jun 21 '18 at 15:59
  • OK. Then the answer is no. Imagine a partition that has primary replica on _Node1_. This replica is represented by an instance of stateful service and all requests made to this partition are processed by this instance _(this is the part you are getting right)_. Now imagine _Node1_ is overloaded and SF moves primary replica to _Node2_ - previous instance on _Node1_ is destroyed and new instance on _Node2_ is created. Now all incoming requests will be processed by a **new** instance. The thing here is that there are plenty of scenarios when SF can destroy instance of primary replica. – Oleg Karasik Jun 21 '18 at 19:48
  • I understand the fail over part. As long as the primary replica does not change, a single Instance of a stateful service continues to service all incoming remoting calls? If so, all class level vars are essentially static, right? – teeboy Jun 21 '18 at 21:51
  • If you are asking about storing information in these variables to be available between requests _(if we ignore the fact that instance can be destroyed at any moment of time)_ then yes - you can store information in this variables. If you are talking about static as .NET static - then no, these variables aren't static - they are instance. This is important to because depending of the [hosting model](https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-hosting-model) there can be more that one instance of stateful services within single process. – Oleg Karasik Jun 22 '18 at 10:18
  • I did read that article. I still have a question. If I store data in class level member variables, and the same object instance is shared across multiple tcp remoting requests, do I need to use c# "lock" if I need to update the class level vars? [if hosting in "exclusive process" as that seems to be the default in appmanifest.xml when creating new services]. Would this behavior change if hosting in "Shared process"? – teeboy Jun 22 '18 at 14:36
  • If we consider only instance fields and properties of the instance that process requests using SF Remoting then the answer is yes for both "exclusive process" and "shared process" hosting models. You need to synchronize access to instance variables. – Oleg Karasik Jun 22 '18 at 15:13

2 Answers2

1

There are some misconceptions here:

Instances are Different than processes, when you say that you can see the instances on Visual Studio, what you actually seeing are processes. There are cases where you can have multiple instances in a single process, and visual studio will show you only one, but there is actually 5(in your case). This is defined by the SF Hosting Model configuration, where you define how it should behave, I have answered related questions here #1 and here #2

The other is, Stateless services have Instances, Stateful services have Partitions & Replicas. Based on that, what you see are not instances, they are Primary replicas(And maybe secondary if you define the replication > 1). See more details in the docs here

Given the previous details, when you do a partitioning of a service, depending on which hosting model you use, it can create multiple Replicas on the same process or one per process.

There is only one catch on all this, the number of replicas from same partition is limited by 1 per node, to have more info about this, take a look at this issue on github

Regarding your questions:

No mater how many calls the clients make, there are only 5 instances of this class to serve requests from. Are the following statements true?

No!

If you are talking about request to an ASP.NET endpoint or remoting calls, this does not affect instance count, instances are defined by the SF configurations and client requests won't affect it, it will be split among the available instances depending how you balance these requests. The only way these requests will affect the instance count is when you define auto-scaling in the service.

When a new instance\replica of your Stateless\Stateful service is started (Within or Outside the same process), a new object is created. A call to these services might always go to the same object, but not will always be the case, because the SF might re-balance your service and kill these instances to create new ones somewhere else, or in case of failures.

There is also the case for stateful services, the calls only go to the primary replicas, unless you specify a read to a secondary one. In case a secondary is promoted to primary, new calls won't be redirected to the former primary replica. What is quite common to happen or cluster maintenance and application updates.

Any class level member variables in the stateful service are essentially static [since it resolves to a singleton instance on that partition] and therefore require "lock" semantics when updating?

If you use shared hosting model, all static objects will be shared with any instance within the same process.

Since the client always resolves to a the "same" instance of the class all the time for a given partition, the client can re-use the "proxy" instance?

Based on previous answer, yes, any singleton class, will be shared. .

How does this "singleton" model of stateful service creation affect performance and scalability when lot of clients call the same service instance hundreds of times a second. [under heavy load] I understand that there is a setting when configuring the V2 remoting listener where we can specify "MaxConcurrentCalls" via "FabricTransportRemotingListenerSettings".

I didn't understand properly the problem here, maybe the previous answers might clear it for your, if not, leave comments below and I will update it.

Diego Mendes
  • 10,631
  • 2
  • 32
  • 36
  • My question is specifically related to the object instances being created by service fabric runtime. When I debug the app in Visual studio and do "this.GetHashCode()" on a stateful service, it always returns the same hashcode for a particular partition. This was what I was referring to. I am not talking about "replicas". Replicas will probably have their own object instance. – teeboy Jun 19 '18 at 17:24
  • I've added more info to the answer. The use of `GetHashCode()` should not be used to identify if the instances are the same, because two similar objects can generate the same hashcode. – Diego Mendes Jun 22 '18 at 10:02
  • @DiegoMendes Service Fabric base classes doesn't override `GetHashCode()` therefore they use default implementation that will return different value for each object. – Oleg Karasik Jun 22 '18 at 15:15
1

Confirmation from Microsoft. Service fabric indeed creates 1 [ONE] instance of a "reliable service" per partition. This "singleton" instance will service all client remoting/http calls. If a failover happens to a replica, a new instance will be created.

teeboy
  • 408
  • 3
  • 13