Method is not executed from running Task

Question

I am working on ASP.NET MVC project.
I need some time o explain my crazy situation.
I am trying to send push notification from the MVC project to the Android and Apple devices.
The Sending logic for both of them are correct, please do not waste your time considering this

The disaster that I am facing is : the static method in a static class which responsible for sending the notification is not called. (I am not fresh programmer, I have more than 5 years in C# programming) but I can not call a method.

to put you in the context of the problem, this method is called and executed and the notification reached to the devices, when I am executing the code on my local machine (development machine).
The static method is just not called when I publish the MVC project and deploy it to our Server.

How do I know that the method is not called ?
Because I am logging to a text file, and I put a log statement at the first line of method and a log statement before calling the method.
The log which is before calling the method is executed and fleshed to the text file, but the log which is in the start of the static method is not executed !!!!!.

Here is some code, and then I will told you what I tried to solve this.

public interface IHandler<T> where T : IMessage
{
    Task Handle(T args);
}

public class RequestAddedAppMonitorHandler : IHandler<RequestAdded>
{
    public Task Handle(RequestAdded args)
    {
        return Task.Factory.StartNew(() =>
        {
            try
            {
                GoogleNotification notification = CreateAndroidPartnerAppNotification(deviceId);

                // this statment is executed, and the text log file will contains this line
                TracingSystem.TraceInformation("Before Send Google Notification");  

                SendersFacade.PartnerSender.Send(notification).Wait();
            }
            catch (Exception ex)
            {
                TracingSystem.TraceException(ex);
            }
        });
    }

    private GoogleNotification CreateAndroidPartnerAppNotification(string to)
    {
        return new GoogleNotification();    // some initialization and creating for the notification object.
    }
}

Facade class

public static class SendersFacade
{
    public static GoogleNotificationSender ClientSender { get; private set; }
    public static GoogleNotificationSender PartnerSender { get; private set; }
    //public static AppleNotificationSender AppleSender { get; private set; }

    static SendersFacade()
    {
        ClientSender = new GoogleNotificationSender("correct api key");
        PartnerSender = new GoogleNotificationSender("correct api key");
        //AppleSender = some intialization.
    }
}

Google Notification Sending Logic

public class GoogleNotificationSender
{
    private string _authorizationToken;
    private string AuthorizationToken
    {
        get { return _authorizationToken; }
        set
        {
            if (string.IsNullOrEmpty(value))
                throw new InvalidOperationException("authorizationToken must not be null");
            _authorizationToken = value;
        }
    }

    public GoogleNotificationSender(string authorizationToken)
    {
        this.AuthorizationToken = authorizationToken;
    }

    public async Task Send(GoogleNotification notification)
    {
        // ATTENTION PLEASE
        // This method is not called, and the following line is not fleshed to the log file
        TracingSystem.TraceInformation("Inside Send Google notification");

        using (HttpClient client = new HttpClient())
        {
            client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "key=" + AuthorizationToken);

            string json = notification.GetJson();
            StringContent content = new StringContent(json, System.Text.Encoding.UTF8, "application/json");

            using (HttpResponseMessage message = await client.PostAsync("https://fcm.googleapis.com/fcm/send", content))
            {
                message.EnsureSuccessStatusCode();

                string resultAsString = await message.Content.ReadAsStringAsync();
                GoogleNotificationResult result = JsonConvert.DeserializeObject<GoogleNotificationResult>(resultAsString);

                if (result.Failure > 0)
                    throw new Exception($"Sending Failed : {result.Results.FirstOrDefault().Error}");
            }
        }
    }
}

Google Notification class

public class GoogleNotification
{
    [JsonProperty("to")]
    public string To { get; set; }

    [JsonProperty("data")]
    public JObject Data { get; set; }

    [JsonProperty("notification")]
    public JObject Notification { get; set; }

    // some other property which is not used at all

    internal string GetJson()
    {
        return JsonConvert.SerializeObject(this,
            new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });
    }
}

What I tried in the previous three days ?

1- Deploy the DLLs which is for the debug (not the published DLLs, with the Release mode) to the server, and this is did not fix the problem.

2- Make the SendersFacade as not static class, and apply the singletone deisng pattern on it, also did NOT worked.

public class SendersFacade
{
    public static SendersFacade Instance { get; private set; }

    public GoogleNotificationSender ClientSender { get; private set; }
    public GoogleNotificationSender PartnerSender { get; private set; }
    //public static AppleNotificationSender AppleSender { get; private set; }

    static SendersFacade()
    {
        if (Instance != null)
            Instance = new SendersFacade();
    }
    public SendersFacade()
    {
        ClientSender = new GoogleNotificationSender("correct api key");
        PartnerSender = new GoogleNotificationSender("correct api key");
        //AppleSender = some intialization.
    }
}

3- Try to put the logic of the sending inside the Handler class it self, and this worked and I was able to send the notification from the server this was, BUT WHY ,IN THE HELL, THIS FOLLOWING CODE IS WORKING, BUT THE PREVIOUS CODE IS NOT WORKING ??????????

public interface IHandler<T> where T : IMessage
{
    Task Handle(T args);
}

public class RequestAddedAppMonitorHandler : IHandler<RequestAdded>
{
    public Task Handle(RequestAdded args)
    {
        return Task.Factory.StartNew(() =>
        {
            try
            {
                GoogleNotification notification = CreateAndroidPartnerAppNotification(deviceId);

                // this statment is executed, and the text log file will contains this line
                TracingSystem.TraceInformation("Before Send Google Notification");  

                SendersFacade.PartnerSender.Send(notification).Wait();
            }
            catch (Exception ex)
            {
                TracingSystem.TraceException(ex);
            }
        });
    }

    private GoogleNotification CreateAndroidPartnerAppNotification(string to)
    {
        return new GoogleNotification();    // some initialization and creating for the notification object.
    }

    private void Send(GoogleNotification notification, string authorizationToken)
    {
        TracingSystem.TraceInformation("Inside Send Google notification");

        using (HttpClient client = new HttpClient())
        {
            client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "key=" + authorizationToken);

            string json = notification.GetJson();
            StringContent content = new StringContent(json, System.Text.Encoding.UTF8, "application/json");

            using (HttpResponseMessage message = client.PostAsync("https://fcm.googleapis.com/fcm/send", content).Result)
            {
                message.EnsureSuccessStatusCode();

                string resultAsString = message.Content.ReadAsStringAsync().Result;
                GoogleNotificationResult result = JsonConvert.DeserializeObject<GoogleNotificationResult>(resultAsString);

                if (result.Failure > 0)
                    throw new Exception($"Sending Failed : {result.Results.FirstOrDefault().Error}");
            }
        }
    }
}

Just adding the logic of the send method to the RequestAddedAppMonitorHandler class solved the problem, BUT I do not want to do that, and why this is happened in the first place ?? It is just calling a method.

3- try to make the send method serial method (not using the async), and it is also did NOT worked

public void Send(GoogleNotification notification)
{
    TracingSystem.TraceInformation("Inside Send Google notification");

    using (HttpClient client = new HttpClient())
    {
        client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "key=" + AuthorizationToken);

        string json = notification.GetJson();
        StringContent content = new StringContent(json, System.Text.Encoding.UTF8, "application/json");

        using (HttpResponseMessage message = client.PostAsync(BASE_URL, content).Result)
        {
            message.EnsureSuccessStatusCode();

            string resultAsString = message.Content.ReadAsStringAsync().Result;
            GoogleNotificationResult result = JsonConvert.DeserializeObject<GoogleNotificationResult>(resultAsString);

            if (result.Failure > 0)
                throw new Exception($"Sending Failed : {result.Results.FirstOrDefault().Error}");
        }
    }
}

NOTE1 : I noticed that I am getting a problem on the server (not appeared at all on my local machine) which is the Application Pool which is specific to this website is stopped frequently, and this is causing the 503 service unavailable when requesting the website.

NOTE 2 : I suspect the most possible cause of this is problem is threads. but I can not reach definite solution

NOTE 3 : Please do not consider that there is answer for this question, it did not helped me at all.

I am working on this from three days, And I am really hopeless, any ideas Thanks.

---

Update the answer of @Nkosi is really helpful, at least I am now know what is the problem, I decided to go synchronous all the way. and to avoid mixing the async/await with the Blocking calles.

so here is the result which I reached

public class RequestAddedAppMonitorHandler : IHandler<RequestAdded>
{
    public Task Handle(RequestAdded args)
    {
        return Task.Factory.StartNew(() =>
        {
            try
            {
                if (deviceOS.Value == DeviceOSEnum.Android.ToString())
                {
                   GoogleNotification notification = CreateAndroidUpdateRequestMessage(args.CustomerRequest, deviceId.Value, notificationString.Title_RequestStared, message);
                   SendGoogleNotification(notification, "some id");
                }
                else if (deviceOS.Value == DeviceOSEnum.IOS.ToString())
                {
                   AppleNotification notification = CreateAppleNotification(deviceId.Value, notificationString.Title_RequestStared, message);
                   AppleNotificationSender sender = new AppleNotificationSender();
                   sender.SendAppleNotification(notification);
                }
            }
            catch (Exception ex)
            {
                TracingSystem.TraceException(ex);
            }
        });
    }

and the AppleNotificationSender class

public class AppleNotificationSender
{
    private TcpClient client;
    private string host = "gateway.push.apple.com";
    private int port = 2195;
    private X509Certificate2 certificate;

    public AppleNotificationSender()
    {
        string path = HostingEnvironment.MapPath("~/Certificates.p12");
        certificate = new X509Certificate2(path, "some correct password");
    }

    private void SetSocketKeepAliveValues(Socket socket, int KeepAliveTime, int KeepAliveInterval)
    {
        //KeepAliveTime: default value is 2hr
        //KeepAliveInterval: default value is 1s and Detect 5 times

        uint dummy = 0; //lenth = 4
        byte[] inOptionValues = new byte[System.Runtime.InteropServices.Marshal.SizeOf(dummy) * 3]; //size = lenth * 3 = 12

        BitConverter.GetBytes((uint)1).CopyTo(inOptionValues, 0);
        BitConverter.GetBytes((uint)KeepAliveTime).CopyTo(inOptionValues, System.Runtime.InteropServices.Marshal.SizeOf(dummy));
        BitConverter.GetBytes((uint)KeepAliveInterval).CopyTo(inOptionValues, System.Runtime.InteropServices.Marshal.SizeOf(dummy) * 2);
        // of course there are other ways to marshal up this byte array, this is just one way
        // call WSAIoctl via IOControl

        // .net 3.5 type
        socket.IOControl(IOControlCode.KeepAliveValues, inOptionValues, null);
    }

    private bool SocketCanWrite(SslStream stream)
    {
        if (client == null)
            return false;

        if (stream == null || !stream.CanWrite)
            return false;

        if (!client.Client.Connected)
            return false;

        return client.Client.Poll(1000, SelectMode.SelectWrite);
    }

    private void Connect()
    {
        try
        {
            if (client == null)
                client = new TcpClient();

            client.Connect(host, port);

            //Set keep alive on the socket may help maintain our APNS connection
            try { client.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.KeepAlive, true); }
            catch { }

            // Really not sure if this will work on MONO....
            // This may help windows azure users
            try
            {
                SetSocketKeepAliveValues(client.Client, (int)TimeSpan.FromMinutes(20).TotalMilliseconds, (int)TimeSpan.FromSeconds(30).TotalMilliseconds);
            }
            catch { }
        }
        catch (Exception ex)
        {
            throw new Exception("Failed to Connect, check your firewall settings!", ex);
        }
    }

    public void SendAppleNotification(AppleNotification notification)
    {
        SslStream stream = null;
        try
        {
            Connect();

            stream = new SslStream(client.GetStream(),
                false,
                (sender, cert, chain, policyErrors) => true,
                (sender, targetHost, localCerts, remoteCert, acceptableIssuers) => certificate);

            try
            {
                X509CertificateCollection collection = new X509CertificateCollection();
                collection.Add(certificate);
                stream.AuthenticateAsClient(host, collection, System.Security.Authentication.SslProtocols.Tls, false);
            }
            catch (System.Security.Authentication.AuthenticationException ex)
            {
                throw new Exception("SSL Stream Failed to Authenticate as Client", ex);
            }

            if (!stream.IsMutuallyAuthenticated)
                throw new Exception("SSL Stream Failed to Authenticate", null);

            if (!stream.CanWrite)
                throw new Exception("SSL Stream is not Writable", null);

            if (!SocketCanWrite(stream))
                Connect();

            byte[] data = notification.ToBytes();
            stream.Write(data, 0, data.Length);
            //TracingSystem.TraceInformation("Write to stream ended.");
        }
        catch (Exception)
        {
            TracingSystem.TraceError("Error in sending Apple notification");
            throw;
        }
        finally
        {
            try { stream?.Close(); } catch { }
            try { stream?.Dispose(); } catch { }
            try { client?.Client?.Shutdown(SocketShutdown.Both); } catch { }
            try { client?.Client?.Dispose(); } catch { }
            try { client?.Close(); } catch { }
            client = null;
        }
    }
}

Now I solved the deadlock problem, but I got another problem. When sending apple notification, the MVC action which trigger this Handle method is called twice, and this will cause an Business Rule exception (normal thing if this Action triggered twice). and the Apple notification is not reached at all.
Note : when I am debugging the code of sending the Apple Notification on my local machine, every thing is good, and the notification reached, and the Action called just for once, the previous described problem appears just when after deploy this code to the server.
Note : This problem does not appear when sending the Google notification at all.

By the way here is the triggering of the Handle method

public class MessageBus : ICommandSender
{
    public static MessageBus Instance { get; private set; }

    private MessageBus()
    {
        handlers = new List<Delegate>();
    }

    static MessageBus()
    {
        if (Instance == null)
            Instance = new MessageBus();
    }

    private List<Delegate> handlers;

    public void Send<T>(T command) where T : ICommand
    {
        List<Task> tasks = new List<Task>();
        foreach (Func<T, Task> handle in handlers.OfType<Func<T, Task>>())
        {
            try { tasks.Add(handle(command)); }
            catch (Exception ex) { TracingSystem.TraceException(ex); }
        }

        try { Task.WaitAll(tasks.ToArray()); }
        catch (BusinessRuleException buzEx) { TracingSystem.TraceException(buzEx); throw buzEx; }
        catch (Exception ex) { TracingSystem.TraceException(ex); }
    }     
}

Answer 1

Looks like you've got a deadlock. You need to read about synchronization context and ConfigureAwait.

I'd recommend you to use:

await SendersFacade.PartnerSender.SendAsync(notification);

Instead of:

SendersFacade.PartnerSender.Send(notification).Wait();

UPD:

If you can't make your Send method async you need to add ConfigureAwait(false) to awaitable methods:

await client.PostAsync("https://fcm.googleapis.com/fcm/send", content).ConfigureAwait(false);

await message.Content.ReadAsStringAsync().ConfigureAwait(false);

This allows you to avoid deadlock.

Answer 2

BUT WHY, THIS FOLLOWING CODE IS WORKING, BUT THE PREVIOUS CODE IS NOT WORKING?

The working code works because it is all being called synchronously and is not mixing async/await and blocking calls.

In the previous code you are mixing async/await with blocking calls like .Result or .Wait() which can lead to deadlocks. You either go async all the way through or synchronous all the way through.

I suggest you refactor GoogleNotificationSender, making sure it is async all the way through

public class GoogleNotificationSender {
    private HttpClient client;
    private string authorizationToken;

    public GoogleNotificationSender(string authorizationToken) {
        this.AuthorizationToken = authorizationToken;
    }

    private string AuthorizationToken {
        get { return authorizationToken; }
        set {
            if (string.IsNullOrEmpty(value))
                throw new InvalidOperationException("authorizationToken must not be null");
            authorizationToken = value;
        }
    }

    private HttpClient Client {
        get {
            if (client == null) {
                client = new HttpClient();
                client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "key=" + AuthorizationToken);
            }
            return client;
        }
    }

    public async Task SendAsync(GoogleNotification notification) {
        TracingSystem.TraceInformation("Inside Send Google notification");

        var json = notification.GetJson();
        var content = new StringContent(json, System.Text.Encoding.UTF8, "application/json");
        var requestUri = "https://fcm.googleapis.com/fcm/send";

        using (var message = await Client.PostAsync(requestUri, content)) {
            message.EnsureSuccessStatusCode();

            var result = await message.Content.ReadAsAsync<GoogleNotificationResult>();
            if (result.Failure > 0)
                throw new Exception($"Sending Failed : {result.Results.FirstOrDefault().Error}");
        }
    }
}

Note the renaming of Send to SendAsync to properly express intent. On a separate note, try not to create a new HttpClient on each call. That can have side effects but it is outside of the scope of this question and answer. There are already many answers on SO explaining this.

Next make sure the Handler is also properly implemented as async

public class RequestAddedAppMonitorHandler : IHandler<RequestAdded> {
    public async Task Handle(RequestAdded args) {
        try {
            string deviceId = args.DeviceId;//This is an assumption here
            var notification = CreateAndroidPartnerAppNotification(deviceId);

            // this statment is executed, and the text log file will contains this line
            TracingSystem.TraceInformation("Before Send Google Notification");

            await SendersFacade.PartnerSender.SendAsync(notification);
        } catch (Exception ex) {
            TracingSystem.TraceException(ex);
        }
    }

    private GoogleNotification CreateAndroidPartnerAppNotification(string to) {
        // some initialization and creating for the notification object.
        return new GoogleNotification() {
            To = to
        };
    }
}

Finally try to make sure that there are no blocking calls being made higher up in the call stack as this would just put you right back into the deadlocking issue you experienced. ie: what ever is calling Task IHandler<T>.Handle(T args) should not mix async and blocking calls.

If having trouble fully understanding async/await, you should really consider reading

Async/Await - Best Practices in Asynchronous Programming

to get a better understanding of the topic.

Answer 3

I'd personally recommend giving PushSharp a go. It provides a great brokered async solution for pushing notifications to iOS, Android, Chrome and Windows Phone.

Found it a lot easier to use myself and report on failed attempts to push. All open source from https://github.com/Redth/PushSharp or through NuGet

Answer 4

I think this problem is you've get a return type of async Task on your call here:

public async Task Send(GoogleNotification notification)

...but you never actually start that task with Task.Run. You DO call .Wait() on it here:

SendersFacade.PartnerSender.Send(notification).Wait();

...but that's not how it works, you actually have to start the task in order to wait for it, it'll just wait there forever like that.

If you change your Send method to have a signature and body like this, it'll work:

    public Task Send(GoogleNotification notification)
    {
        return Task.Run(()=>
        {
            TracingSystem.TraceInformation("Inside Send Google notification");

            using (HttpClient client = new HttpClient())
            {
                client.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "key=" + AuthorizationToken);

                string json = notification.GetJson();
                StringContent content = new StringContent(json, System.Text.Encoding.UTF8, "application/json");

                using (HttpResponseMessage message =client.PostAsync("https://fcm.googleapis.com/fcm/send", content).Result)
                {
                    message.EnsureSuccessStatusCode();

                    string resultAsString = message.Content.ReadAsStringAsync().Result;
                    GoogleNotificationResult result = JsonConvert.DeserializeObject<GoogleNotificationResult>(resultAsString);

                    if (result.Failure > 0)
                        throw new Exception($"Sending Failed : {result.Results.FirstOrDefault().Error}");
                }
            }
        });
    }

Note that I've also removed the await keywords inside the body of the method - I wanted to keep it simple and run synchronously inside the parent Task, after all we're async overall so it doesn't make a difference inside the body.

Here's a complete example, whack it into a console app and give it a go...

    static void Main(string[] args)
    {
        Console.WriteLine("Press any key to run.");
        Example thing = new Example();
        while (Console.ReadKey() != null)
            thing.Send();
    }

    class Example
    {
        Task<DisposableThing> DoTask()
        {
            return Task.Run(() => { Console.WriteLine("DoTask()"); return new DisposableThing(); });
        }
        Task<DisposableThing> DoTaskWillNotWork()
        {
            return new Task<DisposableThing>(() => { Console.WriteLine("DoTaskWillNotWork()"); return new DisposableThing(); });
        }

        async Task<DisposableThing> DoAsync()
        {
            Func<DisposableThing> action = new Func<DisposableThing>(() =>
            {
                Console.WriteLine("DoAsync()");
                return new DisposableThing();
            });
            return await Task.Run(action);
        }

        public Task Send()
        {
            return Task.Run(() =>
            {
                using (DisposableThing client = new DisposableThing())
                {
                    using (DisposableThing message = DoAsync().Result)
                    {
                        DisposableThing resultAsString = DoTask().Result;
                        DisposableThing resultAsString2 = DoTaskWillNotWork().Result;
                    }
                }
            });
        }
    }

    class DisposableThing : IDisposable
    {
        public void Dispose()
        {
            //not much to do
        }
    }

Hope this helps!

Answer 5

I sense deadlocks are at blame here, examine the line HttpResponseMessage message = client.PostAsync("https://fcm.googleapis.com/fcm/send", content).Result with respect to https://blog.stephencleary.com/2012/07/dont-block-on-async-code.html

Combination of async and await with calling .Wait() methods and .Result properties instead of awaiting a task from within an async Task method all the way can result in deadlocks, as described here. https://blog.stephencleary.com/2012/07/dont-block-on-async-code.html

Btw, I do not think your usage of Task.Factory.StartNew is correct. Is there any particular reason for trying to manipulate threads (or tasks) instead of leaving that on the framework. Is there any problem with the following code?

public class RequestAddedAppMonitorHandler : IHandler<RequestAdded>
{
    public async Task Handle(RequestAdded args)
    {
        try
        {
            GoogleNotification notification = CreateAndroidPartnerAppNotification(deviceId);

            // this statment is executed, and the text log file will contains this line
            TracingSystem.TraceInformation("Before Send Google Notification");  

            await SendersFacade.PartnerSender.Send(notification);
        }
        catch (Exception ex)
        {
            TracingSystem.TraceException(ex);
        }
    }
}