caching the result from a [n async] factory method iff it doesn't throw

Question

UPDATE: Heavily revised after @usr pointed out I'd incorrectly assumed Lazy<T>'s default thread safety mode was LazyThreadSafetyMode.PublicationOnly...

I want to lazily compute a value via an async Factory Method (i.e. it returns Task<T>) and have it cached upon success. On exception, I want to have that be available to me. I do not however, want to fall prey to the exception caching behavior that Lazy<T> has in its default mode (LazyThreadSafetyMode.ExecutionAndPublication)

Exception caching: When you use factory methods, exceptions are cached. That is, if the factory method throws an exception the first time a thread tries to access the Value property of the Lazy object, the same exception is thrown on every subsequent attempt. This ensures that every call to the Value property produces the same result and avoids subtle errors that might arise if different threads get different results. The Lazy stands in for an actual T that otherwise would have been initialized at some earlier point, usually during startup. A failure at that earlier point is usually fatal. If there is a potential for a recoverable failure, we recommend that you build the retry logic into the initialization routine (in this case, the factory method), just as you would if you weren’t using lazy initialization.

Stephen Toub has an AsyncLazy class and writeup that seems just right:

public class AsyncLazy<T> : Lazy<Task<T>>
{
    public AsyncLazy(Func<Task<T>> taskFactory) :
        base(() => Task.Factory.StartNew(() => taskFactory()).Unwrap())
    { }

    public TaskAwaiter<T> GetAwaiter() { return Value.GetAwaiter(); }
}

however that's effectively the same behavior as a default Lazy<T> - if there's a problem, there will be no retries.

I'm looking for a Task<T> compatible equivalent of Lazy<T>(Func<T>, LazyThreadSafetyMode.PublicationOnly), i.e. it should behave as that is specified:-

Alternative to locking In certain situations, you might want to avoid the overhead of the Lazy object's default locking behavior. In rare situations, there might be a potential for deadlocks. In such cases, you can use the Lazy(LazyThreadSafetyMode) or Lazy(Func, LazyThreadSafetyMode) constructor, and specify LazyThreadSafetyMode.PublicationOnly. This enables the Lazy object to create a copy of the lazily initialized object on each of several threads if the threads call the Value property simultaneously. The Lazy object ensures that all threads use the same instance of the lazily initialized object and discards the instances that are not used. Thus, the cost of reducing the locking overhead is that your program might sometimes create and discard extra copies of an expensive object. In most cases, this is unlikely. The examples for the Lazy(LazyThreadSafetyMode) and Lazy(Func, LazyThreadSafetyMode) constructors demonstrate this behavior.

IMPORTANT

When you specify PublicationOnly, exceptions are never cached, even if you specify a factory method.

Is there any FCL, Nito.AsyncEx or similar construct that might fit in nicely here? Failing this, can anyone see an elegant way to gate the "attempt in progress" bit (I'm OK with each caller making its own attempt in the same way that a Lazy<T>( ..., (LazyThreadSafetyMode.PublicationOnly) does) and yet still have that and the cache management encapsulated neatly?

Answer 1

Disclaimer: This is a wild attempt at refactoring Lazy<T>. It is in no way production grade code.

I took the liberty of looking at Lazy<T> source code and modifying it a bit to work with Func<Task<T>>. I've refactored the Value property to become a FetchValueAsync method since we can't await inside a property. You are free to block the async operation with Task.Result so you can still use the Value property, I didn't want to do that because it may lead to problems. So it's a little bit more cumbersome, but still works. This code is not fully tested:

public class AsyncLazy<T>
{
    static class LazyHelpers
    {
        internal static readonly object PUBLICATION_ONLY_SENTINEL = new object();
    }
    class Boxed
    {
        internal Boxed(T value)
        {
            this.value = value;
        }
        internal readonly T value;
    }

    class LazyInternalExceptionHolder
    {
        internal ExceptionDispatchInfo m_edi;
        internal LazyInternalExceptionHolder(Exception ex)
        {
            m_edi = ExceptionDispatchInfo.Capture(ex);
        }
    }

    static readonly Func<Task<T>> alreadyInvokedSentinel = delegate
    {
        Contract.Assert(false, "alreadyInvokedSentinel should never be invoked.");
        return default(Task<T>);
    };

    private object boxed;

    [NonSerialized]
    private Func<Task<T>> valueFactory;

    [NonSerialized]
    private object threadSafeObj;

    public AsyncLazy()
        : this(LazyThreadSafetyMode.ExecutionAndPublication)
    {
    }
    public AsyncLazy(Func<Task<T>> valueFactory)
                : this(valueFactory, LazyThreadSafetyMode.ExecutionAndPublication)
    {
    }

    public AsyncLazy(bool isThreadSafe) :
                this(isThreadSafe ?
                     LazyThreadSafetyMode.ExecutionAndPublication :
                     LazyThreadSafetyMode.None)
    {
    }

    public AsyncLazy(LazyThreadSafetyMode mode)
    {
        threadSafeObj = GetObjectFromMode(mode);
    }

    public AsyncLazy(Func<Task<T>> valueFactory, bool isThreadSafe)
                : this(valueFactory, isThreadSafe ? LazyThreadSafetyMode.ExecutionAndPublication : LazyThreadSafetyMode.None)
    {
    }

    public AsyncLazy(Func<Task<T>> valueFactory, LazyThreadSafetyMode mode)
    {
        if (valueFactory == null)
            throw new ArgumentNullException("valueFactory");

        threadSafeObj = GetObjectFromMode(mode);
        this.valueFactory = valueFactory;
    }

    private static object GetObjectFromMode(LazyThreadSafetyMode mode)
    {
        if (mode == LazyThreadSafetyMode.ExecutionAndPublication)
            return new object();
        if (mode == LazyThreadSafetyMode.PublicationOnly)
            return LazyHelpers.PUBLICATION_ONLY_SENTINEL;
        if (mode != LazyThreadSafetyMode.None)
            throw new ArgumentOutOfRangeException("mode");

        return null; // None mode
    }

    public override string ToString()
    {
        return IsValueCreated ? ((Boxed) boxed).value.ToString() : "NoValue";
    }

    internal LazyThreadSafetyMode Mode
    {
        get
        {
            if (threadSafeObj == null) return LazyThreadSafetyMode.None;
            if (threadSafeObj == (object)LazyHelpers.PUBLICATION_ONLY_SENTINEL) return LazyThreadSafetyMode.PublicationOnly;
            return LazyThreadSafetyMode.ExecutionAndPublication;
        }
    }
    internal bool IsValueFaulted
    {
        get { return boxed is LazyInternalExceptionHolder; }
    }

    public bool IsValueCreated
    {
        get
        {
            return boxed != null && boxed is Boxed;
        }
    }

    public async Task<T> FetchValueAsync()
    {
        Boxed boxed = null;
        if (this.boxed != null)
        {
            // Do a quick check up front for the fast path.
            boxed = this.boxed as Boxed;
            if (boxed != null)
            {
                return boxed.value;
            }

            LazyInternalExceptionHolder exc = this.boxed as LazyInternalExceptionHolder;
            exc.m_edi.Throw();
        }

        return await LazyInitValue().ConfigureAwait(false);
    }

    /// <summary>
    /// local helper method to initialize the value 
    /// </summary>
    /// <returns>The inititialized T value</returns>
    private async Task<T> LazyInitValue()
    {
        Boxed boxed = null;
        LazyThreadSafetyMode mode = Mode;
        if (mode == LazyThreadSafetyMode.None)
        {
            boxed = await CreateValue().ConfigureAwait(false);
            this.boxed = boxed;
        }
        else if (mode == LazyThreadSafetyMode.PublicationOnly)
        {
            boxed = await CreateValue().ConfigureAwait(false);
            if (boxed == null ||
                Interlocked.CompareExchange(ref this.boxed, boxed, null) != null)
            {
                boxed = (Boxed)this.boxed;
            }
            else
            {
                valueFactory = alreadyInvokedSentinel;
            }
        }
        else
        {
            object threadSafeObject = Volatile.Read(ref threadSafeObj);
            bool lockTaken = false;
            try
            {
                if (threadSafeObject != (object)alreadyInvokedSentinel)
                    Monitor.Enter(threadSafeObject, ref lockTaken);
                else
                    Contract.Assert(this.boxed != null);

                if (this.boxed == null)
                {
                    boxed = await CreateValue().ConfigureAwait(false);
                    this.boxed = boxed;
                    Volatile.Write(ref threadSafeObj, alreadyInvokedSentinel);
                }
                else
                {
                    boxed = this.boxed as Boxed;
                    if (boxed == null) // it is not Boxed, so it is a LazyInternalExceptionHolder
                    {
                        LazyInternalExceptionHolder exHolder = this.boxed as LazyInternalExceptionHolder;
                        Contract.Assert(exHolder != null);
                        exHolder.m_edi.Throw();
                    }
                }
            }
            finally
            {
                if (lockTaken)
                    Monitor.Exit(threadSafeObject);
            }
        }
        Contract.Assert(boxed != null);
        return boxed.value;
    }

    /// <summary>Creates an instance of T using valueFactory in case its not null or use reflection to create a new T()</summary>
    /// <returns>An instance of Boxed.</returns>
    private async Task<Boxed> CreateValue()
    {
        Boxed localBoxed = null;
        LazyThreadSafetyMode mode = Mode;
        if (valueFactory != null)
        {
            try
            {
                // check for recursion
                if (mode != LazyThreadSafetyMode.PublicationOnly && valueFactory == alreadyInvokedSentinel)
                    throw new InvalidOperationException("Recursive call to Value property");

                Func<Task<T>> factory = valueFactory;
                if (mode != LazyThreadSafetyMode.PublicationOnly) // only detect recursion on None and ExecutionAndPublication modes
                {
                    valueFactory = alreadyInvokedSentinel;
                }
                else if (factory == alreadyInvokedSentinel)
                {
                    // Another thread ----d with us and beat us to successfully invoke the factory.
                    return null;
                }
                localBoxed = new Boxed(await factory().ConfigureAwait(false));
            }
            catch (Exception ex)
            {
                if (mode != LazyThreadSafetyMode.PublicationOnly) // don't cache the exception for PublicationOnly mode
                    boxed = new LazyInternalExceptionHolder(ex);
                throw;
            }
        }
        else
        {
            try
            {
                localBoxed = new Boxed((T)Activator.CreateInstance(typeof(T)));
            }
            catch (MissingMethodException)
            {
                Exception ex = new MissingMemberException("Missing parametersless constructor");
                if (mode != LazyThreadSafetyMode.PublicationOnly) // don't cache the exception for PublicationOnly mode
                    boxed = new LazyInternalExceptionHolder(ex);
                throw ex;
            }
        }
        return localBoxed;
    }
}

Answer 2

Does this get anywhere near your requirements?

The behaviour falls somewhere between ExecutionAndPublication and PublicationOnly.

While the initializer is in-flight all calls to Value will be handed the same task (which is cached temporarily but could subsequently succeed or fail); if the initializer succeeds then that completed task is cached permanently; if the initializer fails then the next call to Value will create a completely new initialization task and the process begins again!

public sealed class TooLazy<T>
{
    private readonly object _lock = new object();
    private readonly Func<Task<T>> _factory;
    private Task<T> _cached;

    public TooLazy(Func<Task<T>> factory)
    {
        if (factory == null) throw new ArgumentNullException("factory");
        _factory = factory;
    }

    public Task<T> Value
    {
        get
        {
            lock (_lock)
            {
                if ((_cached == null) ||
                    (_cached.IsCompleted && (_cached.Status != TaskStatus.RanToCompletion)))
                {
                    _cached = Task.Run(_factory);
                }
                return _cached;
            }
        }
    }
}

Answer 3

For now, I am using this:

public class CachedAsync<T>
{
    readonly Func<Task<T>> _taskFactory;
    T _value;

    public CachedAsync(Func<Task<T>> taskFactory)
    {
        _taskFactory = taskFactory;
    }

    public TaskAwaiter<T> GetAwaiter() { return Fetch().GetAwaiter(); }

    async Task<T> Fetch()
    {
        if (_value == null)
            _value = await _taskFactory();
        return _value;
    }
}

While it works in my scenario (I don't have multiple triggering threads etc.), it's hardly elegant and doesn't provide thread-safe coordination of either

• a single attempt in progress a la LazyThreadSafetyMode.ExecutionAndPublication OR
• a stable result after >= 1 success a la LazyThreadSafetyMode.PublicationOnly

Answer 4

Version as I am using based on @LukeH's answer:

// http://stackoverflow.com/a/33872589/11635
public class LazyTask
{
    public static LazyTask<T> Create<T>(Func<Task<T>> factory)
    {
        return new LazyTask<T>(factory);
    }
}

/// <summary>
/// Implements a caching/provisioning model we can term LazyThreadSafetyMode.ExecutionAndPublicationWithoutFailureCaching
/// - Ensures only a single provisioning attempt in progress
/// - a successful result gets locked in
/// - a failed result triggers replacement by the first caller through the gate to observe the failed state
///</summary>
/// <remarks>
/// Inspired by Stephen Toub http://blogs.msdn.com/b/pfxteam/archive/2011/01/15/asynclazy-lt-t-gt.aspx
/// Implemented with sensible semantics by @LukeH via SO http://stackoverflow.com/a/33942013/11635
/// </remarks>
public class LazyTask<T>
{
    readonly object _lock = new object();
    readonly Func<Task<T>> _factory;
    Task<T> _cached;

    public LazyTask(Func<Task<T>> factory)
    {
        if (factory == null) throw new ArgumentNullException("factory");
        _factory = factory;
    }

    /// <summary>
    /// Allow await keyword to be applied directly as if it was a Task<T>. See Value for semantics.
    /// </summary>
    public TaskAwaiter<T> GetAwaiter()
    {
        return Value.GetAwaiter();
    }

    /// <summary>
    /// Trigger a load attempt. If there is an attempt in progress, take that. If preceding attempt failed, trigger a retry.
    /// </summary>
    public Task<T> Value
    {
        get
        {
            lock (_lock) 
                if (_cached == null || BuildHasCompletedButNotSucceeded())
                    _cached = _factory();
            return _cached;
        }
    }

    bool BuildHasCompletedButNotSucceeded()
    {
        return _cached.IsCompleted && _cached.Status != TaskStatus.RanToCompletion;
    }
}