1

I read about continuous deployment of service fabric with VSTS. I need a help/suggestion in this scenario where

  • I have a group of services deployed by Continuous deployment in Azure
  • Now I upgrade one service, I know that when this upgrade fails, service fabric rolls back to the previous state. Lets assume the upgrade is successful, Now I run integration tests (as a part of build definition pipeline) and it failed, in this case how to rollback this particular service alone, so that the other services remains unaffected and roll back should be automated, there should not be any manual intervention

Example: -

  1. Push your code
  2. Upgrade the deployment of service A where group of services are running
  3. Perform integration tests
  4. On failure, Rollback service upgrade of A and on success, continue to upgrade other nodes

Can this be achieved fully automated in VSTS?

I referred this link: https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-application-upgrade#rolling-upgrades-overview

Hari Priya Thangavel
  • 487
  • 7
  • 16

2 Answers2

2

The rollback API can only be used to rollback the current in-progress upgrade that is rolling forward to new version, you can roll back a completed upgrade through API, so you can’t do it in VSTS build/release.

Based on this thread: Staging slot and vip-swap, you can create an instance for a new application version.

starian chen-MSFT
  • 33,174
  • 2
  • 29
  • 53
0

You can use the built-in service health monitoring. By implementing custom health monitors (that run your integration tests), you can report 'unhealthy' when they fail during upgrade. You can use this information to have SF automatically roll back the upgrade. (By configuring health thresholds.) You can also control upgrades manually, for example by using PowerShell Start-ServiceFabricApplicationUpgrade.

Scott Hanselman gave a good presentation about this in action here. Another example here.

LoekD
  • 11,402
  • 17
  • 27
  • Thanks for the quick answer, Can Custom health checks include functional test case ? – Hari Priya Thangavel Feb 13 '18 at 15:18
  • You can use (any) code to check things, and use the result to populate a health report object, e.g. `DeployedServicePackageHealthReport`. (there's no special test runner) – LoekD Feb 13 '18 at 15:24
  • @LoekD, but isn't there a possibility that after step #2 completes (successful deployment of the new package), SF will move on to the next Update Domain and the new code could execute before the health checks (integration tests) have completed? Or can the upgrade specify that some health checks must pass before the upgrade is considered "complete"? – PatrickSteele Feb 13 '18 at 16:44
  • Health checks are performed before moving to the next upgrade domain. You specify how tolerant your platform is to unhealthiness. https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-health-introduction#health-policies – LoekD Feb 13 '18 at 19:26
  • Thanks @LoekD for the information. The page you referenced led me to [this page](https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-application-upgrade) which explains the upgrade process more and talks about the health checks. – PatrickSteele Feb 14 '18 at 02:43
  • @LoekD Thank you so much for clarifying. I will try implementing this!!!!! – Hari Priya Thangavel Feb 15 '18 at 03:38
  • @HariPriyaThangavel Do you solve this issue after trying it. – starian chen-MSFT Feb 19 '18 at 07:09
  • @starianchen-MSFT I am currently trying to deploy a external watchdog as service fabric service, which will be doing the integration/functional testing (end to end service flow). With this I will know whether upgrade waits and runs the watchdog's custom health check and rollback if it fails. This I am trying as a POC and hope that will answer all of my questions. – Hari Priya Thangavel Feb 19 '18 at 09:37