Is it possible to configure Guava Cache (or other library) behaviour to be: If time to reload, return previous entry, reload in background (see specs)

Question

I'd like to have a cache that works like this:

A. If request is not cached: load and return results.
B. If request is cached, has not expired: return results.
C. If request is cached, has expired: return old results immediately, start to reload results (async)
D. If request is cached, has expired, reload is already running: return old results immediately.
E. If reloading fails (Exception): continue to return previous successful load results to requests.

(After a failed reload (case E), next request is handled following case C.)

(If case A ends in Exception, Exception is thrown)

Does anyone know an existing implementation, or will I have to implement it myself?

I think what you are looking for is [asyncReloading](http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/cache/CacheLoader.html#asyncReloading(com.google.common.cache.CacheLoader,%20java.util.concurrent.Executor)). Another alternative is to implement it by yourself with a wrapper. — oshai, Nov 17 '15 at 07:53
@oshai Thank you! This seems to be a new feature I have not noticed before. Looks like it doesn't quite handle exceptions like I'd like, but I can develop a wrapper to handle that. I'll return to this once I have had time to fiddle around. — Esko Piirainen, Nov 17 '15 at 11:43

score 2 · Answer 1 · answered Nov 23 '15 at 04:24

In cache2k I implemented exactly the behavior you described above. Here is how you build a cache with these features:

  CacheBuilder.newCache(Key.class, Value.class)
    .name("myCache")
    .source(new YourSourceImplementation())
    .backgroundRefresh(true)
    .suppressExceptions(true)
    .maxSize(7777) // maximum entries in the cache
    .expiryDuration(60, TimeUnit.SECONDS)
    .exceptionExpiryDuration(15, TimeUnit.SECONDS)
    .build();

The exipryDuration is the duration the value is considered valid after it war inserted or modified. The separate setting for exceptionExpiryDuration is the time until the next refresh is tried after an exception happens.

If an exception happens, but there is no valid entry, the exception is cached and rethrown for the exceptionExpiryDuration time.

You can also dynamically compute the expiry duration, e.g. based on the exception type. Some more information is in the blog entry About caching exceptions

With backgroundRefresh an entry is refreshed after it is expired. When no access happens after an refresh within the expiry time, the entry will not get refreshed any more and then eventually evicted.

Unfortunately, I am really behind in documenting all these useful features properly. If you have any more questions you can use the tag cache2k. If you like some improvements, open an issue on GitHub.

The code works well in our production applications for about a year now. Actually, suppressExceptions is always the default. It helps very well, e.g. if there is a short network outage.

BTW: Meanwhile, I subsume these semantics under the term cache resiliency.

It has been some time :) How'd you compare cache2k to caffeine? I was back at 15 hesitant to go with chache2k because it was not so popular (=read: maintained, scrutinized) as guava for example — Esko Piirainen, May 20 '21 at 09:33

score 1 · Accepted Answer · answered Jan 07 '20 at 10:29

https://github.com/ben-manes/caffeine

Caffeine provides exactly the behavior I want out of the box using refreshAfterWrite:

LoadingCache<K, V> cache = Caffeine.newBuilder()
   .refreshAfterWrite(expireTime, timeUnit)
   .maximumSize(maxCountOfItems)
   .build(k->loader.load(k));

Is it possible to configure Guava Cache (or other library) behaviour to be: If time to reload, return previous entry, reload in background (see specs)

2 Answers2

Linked