2

We have an asp.net web site that is deployed on several IIS servers. The site is compile-on-demand as opposed to a pre-compiled web application.

Normally deployments go fine but every now and again we get a 401 for one of the deployed pages on one of the servers. There is nothing special about which page or which server apart from the fact that it's generally the higher traffic pages that it happens to.

The only way to rectify this is to deploy the same page again.

The ACLs look fine on the files themselves so the thought is that there is a file locking issue in the Temporary ASP.NET Files folder when the specific page is re-compiled.

Has anyone seen this before or have any suggestions how to avoid this?

Note: This only seems to have happened since we moved to .net 4.0

As far as I can tell we are getting a 401.3 Denied by resource ACL http://support.microsoft.com/kb/907273 But I have not been able to confirm this.

Chris Simpson
  • 7,821
  • 10
  • 48
  • 68

4 Answers4

1

Those kinds of locks have always been a problem with live site deployment. The reason it's hard to replicate is because you are mid-request when copying/compiling on the server, and this ends up confusing IIS.

We operate a Blue/Green deployment strategy on a 4 tier architecture which has a web site over 4 servers at the top tier. Due to the complexity the architecture introduced for deployments, we needed a way to deploy without disturbing any traffic to the "live" site. Following Fowler's advice, but not quite in the same way, we came up with a solution that means we have 2 sites on each server (a blue and a green, or in our case site A and site B). The live site has the appropriate host header, and once we have deployed and tested to the non-live site, we then flip the headers of the 2 sites so that what was once live is now the non-live site, and vice-versa. The effect is, a robust deployment that can be done in hours and with the highest level of confidence.

This of course complicates your configuration and deployment slightly, but it's worth the effort. I guess it kind of goes without saying that you want to script both the deployment, and the host header swapping.

pms1969
  • 3,354
  • 1
  • 25
  • 34
  • When you say "always been a problem", is that just in your own experience or is there any documentation to back that up? I'm really interested in finding out exactly what is being locked and why. If there is a bug in IIS I don't want to go through the considerable effort of changing our deployment process without first knowing exactly what the issue is. – Chris Simpson Apr 26 '12 at 16:11
  • Definitely my own experience, and that didn't have anything to do with our move to Blue/Green. Completely different and sad story that one. We do however have a couple of other sites, and when deployments are done live during business hours, we sometimes have trouble, due to heavy load at the time the deployment is done, and this causes the app pool to stop completely. Similar to this http://blogs.msdn.com/b/tess/archive/2006/08/02/asp-net-case-study-lost-session-variables-and-appdomain-recycles.aspx – pms1969 Apr 27 '12 at 06:55
0

When i deploy to a server i bring the site down for a minute (or however long the deployment takes) - it may be down anyway during this time as pages are recompiled so it is not too much of a hit. You can do this by creating a file in the root of the app called app_offline.aspx (it needs at least 512 characters in length) once that file is created you can then copy the resources ot the folder knowing there will not be any locking issues. then when the copy is complete remove the app_offline file.

Symeon Breen
  • 1,531
  • 11
  • 25
0

For those that want to achieve a .net website deployment without these issues, one option is to copy the new website files into a new folder first ( not the active website). Then you just change IIS to point to the new folder after all copying is complete.

This can be done in a single server environment for those of us on more limited resources without multiple servers per website.

At my work we write power shell scripts to deploy websites. The powers shell script creates a new directory with a time stamp, copies the new deployment there, then tells IIS to point the website to the new directory (leaving the old directories "orphaned" but still there).

If we really messed something up, we can simply revert by pointing IIS back at the previous date stamp directory. Otherwise if everything tests ok, we can delete the old folder.

This technique works well because you are never writing over a file while it is in use. However it still results in zero downtime. The only effect you will see is the normal .net "warm up" that occurs anytime you change the code behinds or assemblies.

BenSwayne
  • 16,810
  • 3
  • 58
  • 75
0

I had several answers suggesting a new environment to deploy. This is something we have been considering for the long term but it's hard to justify the extra work when we regularly deploy only one or two files without a problem. I was really more interested in finding out what is actually happening and why.

In terms of a workaround, and this might sound obvious after the fact, a simple app_pool recycle solves the permissions issue and is much easier than testing for the issue and redeploying the file until the problem goes away.

Chris Simpson
  • 7,821
  • 10
  • 48
  • 68