28

Overview

We have a multinational website which has localised content for the various countries it serves. This localisation is implemented using standard .Net resource files.

When our web application starts up or recycles under load on the production environment, sometimes it will display the wrong resources for a particular country. E.g. the UK site may show French content.

This continues to happen until the application is restarted.

Detail

The production environment is IIS 8 on Windows Server 2012. The application is implemented in ASP.Net MVC 4.

The application decides which locale it is serving by the incoming URL. So www.mysite.com will be UK English www.mysite.fr will be French etc.

We have an implementation of IHttpModule which is registered via a Web.config. In the Init method of the module, it attaches a handler to the BeginRequest event. In this method, the incoming URL is examined and the thread's CurrentUICulture is set to an appropriate value. en-GB for www.mysite.com, fr-FR for www.mysite.fr etc.

This system works well for the most part. However, sometimes when the application starts up while it is receiving requests, it will consistently serve the wrong content for some of the resource files.

It continues to do this until the application is restarted. It may again restart serving the incorrect content. We have to keep restarting until it is serving the correct content, at which point it will remain stable.

Analysis

We have been able to reproduce this locally on a development PC by throwing requests at the application during startup (using Fiddler). The site was showing German content for some resource files on the UK version of the site.

Having checked the obvious culprits in our code (that the CurrentUICulture is set correctly by the HTTP module and remains correct throughout the processing of the request), we started to look at the resource manager.

With the application started in this incorrect state, we examined the contents of the _resourceSets property on the ResourceManager class. This is a Dictionary keyed on the ISO culture code. Examining the contents of en-GB, we found that it did actually contain the resource strings from the German version of the resource file.

It seems that sometimes, when the site is starting up while receiving requests, the ResourceManager class is loading the wrong resource file for a culture or it is incorrectly classifying the file in its Dictionary.

Has anyone else experienced this kind of behavior and is anyone aware of any workarounds?

Thanks.

Imad Alazani
  • 6,688
  • 7
  • 36
  • 58
Stevo
  • 1,424
  • 11
  • 20
  • Can you post simplifed code of your module and handler? Shouldn't module be enough? – Nenad Jul 25 '13 at 01:18
  • We got the same or a similar problem. We have two projects (A and B), they both got project-specific texts in resx-files in them. Then we got a satellite resource assembly with shared texts. The texts fetched from the project-specific resx-files are always correct, but the texts from the satellite resource assembly sometimes get stuck to English after an app pool recycle. This means that pages sometimes contains some texts in English and some texts in Swedish. The only thing that "helps" is to recycle the app pool until English isn't stuck anymore. – Carl Björknäs Jan 12 '16 at 10:54
  • Moved the local resource files from App_GlobalResources to a normal folder, haven't been able to reproduce the problem after that. http://odetocode.com/Blogs/scott/archive/2009/07/16/resource-files-and-asp-net-mvc-projects.aspx – Carl Björknäs Jan 14 '16 at 13:44

5 Answers5

5

This sounds like you are having thread safety problems in your handler. When you modify the threads current culture, you are modifying it for the current thread that could be processing multiple requests. When the responses are generated, another request could have altered the thread current language giving all responses the same language.

I have a few suggestions you can start off with:

  1. Make sure your handler is NOT reusable. I assume you've made implemented IHttpHandler, return false for the IsReusable property since multiple threads should hit it at the same time and a new instance will be created per request.
  2. Don't use a handler... A handler is not the ideal solution for something like this, The preferred place to set the thread culture is in Application_AcquireRequestState which will be correctly fired for each request without overlapping.
  3. Use a routing handler instead: http://adamyan.blogspot.com/2010/07/addition-to-aspnet-mvc-localization.html

Without seeing the handler you speak of the rest can only be speculation as to why responses are sharing thread cultures. The problem could easily lie in the dictionary you are using as well which is inherently not thread safe.

BrutalDev
  • 6,181
  • 6
  • 58
  • 72
  • 1
    Thanks for the response, but I don't think that's what is happening. If it was the thread culture changing during request processing, I would expect different results every time. But, if the application starts up in this state, the responses stay the same for every request; some resources on a page in German and some in English. I've traced the culture through the request processing, and it's correct throughout. I've looked at the state of .Net ResourceManager class and it's internal state is wrong. Its dictionary of resources against cultures has German resources in it's entry for en-GB. – Stevo Jul 23 '13 at 11:14
  • Is this question the same as what you are experiencing? - http://stackoverflow.com/questions/5816773/currentthread-currentuiculture-is-set-correctly-but-seems-to-be-ignored-by-asp-n – BrutalDev Jul 23 '13 at 11:53
  • No, those symptoms aren't the same. If our site starts up correctly, the localisation works fine. It appears that when starting under load, the resource manager either loads the wrong resources for a culture or incorrectly classifies the ones it's loaded. – Stevo Jul 23 '13 at 12:05
  • 2
    "Under load" is suspicious because that's exactly when the resource manager can build it's content incorrectly when multiple requests are changing the current UI thread culture. Have you tried doing this outside of your own handler in in one of the HttpContext events? You may also be better off warming up the resource manager on Application_Start before any requests are processed, therefore setting it up correctly before it can be flooded with requests. – BrutalDev Jul 23 '13 at 12:21
  • If you are using a custom HttpModule (as specified) you will most certainly run into threading issues since every request goes through the same module and there will be race conditions setting the thread culture. http://forums.asp.net/t/900381.aspx/1. Two requests for different languages will overwrite eachother and both respond with the last set language. The static resource manager is probably getting itself in a knot when all of this is happening. Just try an alternative solution, there are so many 'better' options that an HttpModule. – BrutalDev Jul 23 '13 at 12:47
  • Warming up the resource manager in app start is a good shout. Regarding your next comment, what are the better options to using a HttpModule? Thanks. – Stevo Jul 23 '13 at 13:23
2

We hit the same problem recently. This appears to be a race condition in ResourceManager (or its helper code). I've put together a repro at https://bitbucket.org/onyxmaster/resmanrc. Also, I filed a bug on MS Connect site, at https://connect.microsoft.com/VisualStudio/feedback/details/806505/.

P.S. I'm not sure if this counts as an answer since there is no workaround I know of, but at least now there is a repro and a bug report.

  • Thank you! It bothered my had for a long time, glad I've found your comment. – Ilya Luzyanin Nov 24 '14 at 11:17
  • Any news regarding this? We probably got the same problem. The url to the bug report is not working... – Carl Björknäs Jan 12 '16 at 10:46
  • It appears that they (Microsoft) deleted the issue, but it won't help you much, because while they acknowledged the issue, no fix estimates or workarounds were provided. It was easy for us to fix it, because we access resources via a wrapper, so adding a lock around resource access (HttpContext.GetGlobalResourceObject/GetLocalResourceObject in our case) wasn't that hard. – Aristarkh Zagorodnikov Jan 13 '16 at 13:21
  • @onyXMaster Do you know if this bug is still there under NET 4.5.1, 4.5.2 or .NET 4.6 ? – fstarnaud Mar 23 '16 at 02:12
  • @fstarnaud Unsure, I had no reason to test it (although we're running 4.6.1 now). – Aristarkh Zagorodnikov Mar 24 '16 at 12:38
  • @fstarnaud I did a quick test of the repro I provided, after changing target .NET FW to 4.6.1 -- nothing changed, it still fails both in shared (expected, no one guaranteed it to be thread-safe) and private (the real culprit is here) cases. – Aristarkh Zagorodnikov Mar 24 '16 at 12:42
  • It looks like it's still there... We have reproduced on WebAPI 2.2, framework 4.5.2. – Yan Oreshchenkov Jun 29 '16 at 08:41
1

For anyone else experiencing this problem, I never got a fix for the underlying cause of this problem, but did find a workaround. When the app starts up, I had a routine go through each resource file sequentially and request a resource from each file in each supported language. 'Touching' each file in this way in a single threaded manner seemed to allow all the resources to load correctly every time.

Stevo
  • 1,424
  • 11
  • 20
0

Are you using async MVC actions ? If yes, when you await a call (with ConfigureAwait set to false), the processing thread is reused while it is in progress. the "returning" thread (the thread executing the code after the awaited call) is not the same and may loose all properties set before. ConfigureAwait must be set to false to prevent deadlocks, so there is no immediate solution.

Another thing to check is caching, if you use the [Cache] attrbute or if you cache the resouce manager.

Softlion
  • 12,281
  • 11
  • 58
  • 88
  • We're not currently using async actions, and have turned off all caching to help troubleshoot this issue. – Stevo Jul 29 '13 at 08:56
0

I read this answer from other resources. And would like to wrap the ResourceManager as follows so the new RexourceManager() is not being raced under loaded multi-thread call:

public sealed class LocalizationHandler
{
   [ThreadStatic]
   private static ResourceManager _manager
   private readonly ConcurrentBag<ResourceManager> 
       _localizationIdentityCollection = new ConcurrentBag<ResourceManager>();

   private LocalizationHandler(){}

   public static LocalizationHandler Load(ResourceType source)
   {
      switch (source)
      {
          case typeA:
              //check if resource exist in the concurrent bag or create a new one
           _manager = getManager();
           break;
      }
      return this;
   }

   public string Get(string key)
   {
       return _manager.Get(key)
   }
}

Then you can call like:

LocalizationHandler.Load(ResourceType.TypeA).Get("your resource Key String")

Further more, you can use type safe enum for the resource type with name for resource name, and wrap the resource manager together with the culture info into a private class the save it in the concurrent bag.

appletwo
  • 1,128
  • 1
  • 8
  • 6