2

I am using PushStreamContent to keep a persistent connection to each client. Pushing short heartbeat messages to each client stream every 20 seconds works great with 100 clients, but at about 200 clients, the client first starts receiving it a few seconds delayed, then it doesn't show up at all.

My controller code is

    // Based loosely on https://aspnetwebstack.codeplex.com/discussions/359056
    // and http://blogs.msdn.com/b/henrikn/archive/2012/04/23/using-cookies-with-asp-net-web-api.aspx
    public class LiveController : ApiController
    {
        public HttpResponseMessage Get(HttpRequestMessage request)
        {
            if (_timer == null)
            {
                // 20 second timer
                _timer = new Timer(TimerCallback, this, 20000, 20000);    
            }

            // Get '?clientid=xxx'
            HttpResponseMessage response = request.CreateResponse();
            var kvp = request.GetQueryNameValuePairs().Where(q => q.Key.ToLower() == "clientid").FirstOrDefault();
            string clientId = kvp.Value; 

            HttpContext.Current.Response.ClientDisconnectedToken.Register(
                                                                    delegate(object obj)
                                                                    {
                                                                        // Client has cleanly disconnected
                                                                        var disconnectedClientId = (string)obj;
                                                                        CloseStreamFor(disconnectedClientId);
                                                                    }
                                                                    , clientId);

            response.Content = new PushStreamContent(
                                    delegate(Stream stream, HttpContent content, TransportContext context)
                                    {
                                        SaveStreamFor(clientId, stream);
                                    }
                                    , "text/event-stream");


            return response;
        }

        private static void CloseStreamFor(string clientId)
        {
            Stream oldStream;
            _streams.TryRemove(clientId, out oldStream);
            if (oldStream != null)
                oldStream.Close();
        }

        private static void SaveStreamFor(string clientId, Stream stream)
        {
            _streams.TryAdd(clientId, stream);
        }


        private static void TimerCallback(object obj)
        {
            DateTime start = DateTime.Now;

            // Disable timer
            _timer.Change(Timeout.Infinite, Timeout.Infinite);      

            // Every 20 seconds, send a heartbeat to each client
            var recipients = _streams.ToArray();
            foreach (var kvp in recipients)
            {
                string clientId = kvp.Key;
                var stream = kvp.Value;
                try
                {
                    // ***
                    // Adding this Trace statement and running in debugger caused
                    // heartbeats to be reliably flushed!
                    // ***
                    Trace.WriteLine(string.Format("** {0}: Timercallback: {1}", DateTime.Now.ToString("G"), clientId));

                    WriteHeartBeat(stream);
                }
                catch (Exception ex)
                {
                    CloseStreamFor(clientId);
                }
            }

           // Trace... (this trace statement had no effect) 
            _timer.Change(20000, 20000);      // re-enable timer
        }

        private static void WriteHeartBeat(Stream stream)
        {
            WriteStream(stream, "event:heartbeat\ndata:-\n\n");
        }


        private static void WriteStream(Stream stream, string data)
        {
            byte[] arr = Encoding.ASCII.GetBytes(data);
            stream.Write(arr, 0, arr.Length);
            stream.Flush();
        }

        private static readonly ConcurrentDictionary<string, Stream> _streams = new ConcurrentDictionary<string, Stream>();
        private static Timer _timer;
    }

Could there be some ASP.NET or IIS setting that affects this? I am running on Windows Server 2008 R2.

UPDATE:

Heartbeats are reliably sent if 1) the Trace.WriteLine statement is added, 2) Visual Studio 2013 debugger is attached and debugging and capturing the Trace.WriteLines).

Both of these are necessary; if the Trace.WriteLine is removed, running under the debugger has no effect. And if the Trace.WriteLine is there but the program is not running under the debugger (instead SysInternals' DbgView is showing the trace messages), the heartbeats are unreliable.

UPDATE 2:

Two support incidents with Microsoft later, here are the conclusions:

1) The delays with 200 clients were resolved by using a business class Internet connection instead of a Home connection

2) whether the debugger is attached or not really doesn't make any difference;

3) The following two additions to web.config are required to ensure heartbeats are sent timely, and failed heartbeats due to client disconnecting "uncleanly" (e.g. by unplugging computer rather than normal closing of program which cleanly issues TCP RST) trigger a timely ClientDisconnected callback as well:

<httpRuntime executionTimeout="5" />

<serverRuntime appConcurrentRequestLimit="50000" uploadReadAheadSize="1" frequentHitThreshold="2147483647" />

David Ching
  • 1,903
  • 17
  • 21
  • 1
    Analyzing in the VS debugger along with Reflector, stream.Flush does nothing. The stream passed to the PushStreamContent delegate is a CompleteTaskOnCloseStream, which inherits from DelegatingStream, whose inner stream is HttpResponseStream, which delegates to HttpWriter to perform the Write. HttpWriter buffers the data and doesn't flush. This is confirmed [here](http://www.differentpla.net/content/2012/07/streaming-http-responses-net). [Justin Rebeiro](https://www.justinribeiro.com/chronicle/2014/04/09/sse-asp-net-mvc-double-message/) confirms the flush does not reliably occur. – David Ching Jul 15 '14 at 14:24
  • Is there any solution to this?. The whole point of using Chunked/streaming is to get the data fast to the client, I have a process that may take minutes to get all the data, but at least as some to begin with. Currently ASP.NET decides it will wait until its all done (sometimes). So I might as well have written the entire thing as a synchronous call anyway. – Beau Trepp May 22 '15 at 03:55
  • @user1570690: See above Update 2. Good luck! – David Ching May 25 '15 at 16:49
  • uploadReadAheadSize looks like what I want, at least it lets me control the buffer. I haven't managed to get VS to actually run with that config yet, but it looks like the right option http://www.iis.net/configreference/system.webserver/serverruntime . Thanks :) – Beau Trepp May 26 '15 at 00:15
  • None of those options work for me in my Azure App Serivce; the only thing that works is to dispose of the StreamWriter after I post my message, which gets auto-repaired anyway. Its not nice, but it works – Dan Sep 06 '22 at 20:39

0 Answers0