4

I'm working on a web service to read data from a 3rd party feed, modify it a little and store it, then return it to my clients. It only needs to update from the 3rd party site periodically. It will be running as a WCF service in a web role in Azure.

At first I thought I would just always make a call to my parsefeed method, but make that call return if the last update was too soon...

   public void ParseFeed()
    {
        if (DateTime.Now > lastrun.AddMinutes(1))
        {
         //Fetch updated data into something shared here.
         //thedata is a public static in Global class 
         thedata = fetchdata();
         lastrun=DateTime.Now;            
        }
    }

But I guess that as the fetch can take 1-2s (its a web service) that multiple users will hit that code at once.

from http://support.microsoft.com/default.aspx?scid=kb;en-us;Q312607 Because static members of any class, including an application class, are not thread-safe, the user code must provide appropriate locking for access to static members. This applies to any static member that you add to the application class.

  • I could use locking (not sure how) EDIT: lots of info here

  • I could avoid the static var and use a cache, and put my data in it (but that would get removed at expiration and multiple users would try to fetch data)

  • I could use a cache with a fake item of data in it (basically as a timer) and refresh when that expired - but that would refresh even if nobody was hitting the site. (May also not be thread safe)

  • I can't really use an output cache because the clients query the data I return in a manner that probably makes each request unique: my service sorts and filters according to the request

BTW I am not worried about consistency of results over multiple instances on Azure. Each can fetch their own so I don't need to share state over multiple servers.

I get the feeling there is a simple solution, that I have totally missed. Ideas ?

Community
  • 1
  • 1
Andiih
  • 12,285
  • 10
  • 57
  • 88
  • I spotted this http://stackoverflow.com/questions/39112/what-is-the-best-way-to-lock-cache-in-asp-net#40065 which shows how to manage the locks. This could apply to most of the options, and still leaves the question of whether to use cache, static vars or if there is a simpler mechanism – Andiih Nov 26 '10 at 09:53

2 Answers2

1

Since you probably will be reading from the cache more than you will be writing, I would use a ReaderWriterLockSlim to ensure data can be read by multiple threads simultaneously but not written. I would also make sure that the cached data is immutable to make sure it's not altered by consumers.

Patrik Svensson
  • 13,536
  • 8
  • 56
  • 77
1

From your question, it looks like:

  • It is not acceptable to serve data that's more than a minute out of date
  • It is acceptable if two service instances return different data due to refresh times being out of sync
  • It is acceptable for callers to block for 1-2 seconds while data is refreshed

Assuming these are all true, then the simplest solution is just to use a static variable to store the data, with a lock construct around the whole check/refresh block. I wouldn't even bother trying to do anything clever like the double-check lock pattern; lock contention simply won't be an issue as the time spent in the critical region will pale into insignificance compared to the overhead of operating a web service, except when it is blocking and everybody has to block anyway.

Greg Beech
  • 133,383
  • 43
  • 204
  • 250
  • I'd actually say that once data is about a minute out of date its time to get new data i.e. It would be better to initiate a background non-blocking refresh while serving stale data. On the other hand, if data was say an hour out of date (no use overnight) then I'd need to block fetch and return. – Andiih Nov 26 '10 at 12:42
  • this is the route I went with, although I did implement the lock recheck, as there may be a number of users blocked during the fetch, all of which would then try to fetch otherwise... – Andiih Nov 27 '10 at 15:54