We have built a framework for our background daemons, to help keep them behave consistently, and so we don't have to keep rewriting the same code.
And, of course, ten years later we built a new one.
One of the applications in the old framework is a SignalR signalling app. It hosts a SignalR instance in Microsoft's Owin, and then periodically checks the database for whatever it needed to push, then calls into the SignalR website, which would then use the configured backplane to talk to whatever other webservers were running, which would then push out to the various web clients.
Kind of convoluted, but it works.
And now it's my job to move it to the new background app framework, and I'm getting HTTP 500 errors when I try to start the web app.
The code in the new daemon is pretty much the same as in the old app, and I can't see why it's working in the one and not in the other.
The way both frameworks work is that each daemon has a doWork() function that is called repeatedly. In the old app, there was a busy/wait loop that would call doWork(). In the new app doWork() is called from a System.Timers.Timer ElapsedEventHandler.
We've defined a class to wrap the SignalR instance:
public class SignalRWebApp : IDisposable
{
public readonly string signalRUrl;
private IDisposable webApp;
public SignalRWebApp()
{
this.signalRUrl = String.Format("http://localhost:{0}", getFreePort());
this.webApp = null;
}
private static int getFreePort()
{
var listener = new TcpListener(IPAddress.Loopback, 0);
listener.Start();
var port = ((IPEndPoint) listener.LocalEndpoint).Port;
listener.Stop();
return port;
}
public bool started { get { return this.webApp != null; } }
public void start(string signalRBackplaneConnectionString)
{
if (this.webApp != null)
return;
Action<IAppBuilder> startAction = app =>
{
app.UseCors(CorsOptions.AllowAll);
GlobalHost.DependencyResolver.UseSqlServer(signalRBackplaneConnectionString);
app.MapSignalR();
};
this.webApp = WebApp.Start(this.signalRUrl, startAction);
}
#region IDisposable
[...]
#endregion
}
We're storing an instance of this in a static member, so that the work() function can access it:
public class Utils
{
public static SignalRWebApp signalRWebApp;
}
And then we wrap our startup code in a using(), so that it exists while the app is running. Note - if you look at the SignalRWebApp class above, you'll note that it doesn't actually do anything, until it's started.
using (Utils.signalRWebApp = new SignalRWebApp())
{
// initialize and run the background app
// (this will repeatedly call work() until shutdown is requested)
}
Our work function then starts by starting up the SignalRWebApp, if it's not already running:
public class JobLockDaemon
{
private string signalRUrl;
private IHubProxy ticketLockSignalRHubProxy;
private HubConnection signalRConnection;
public JobLockDaemon()
{
this.checkedSnapshot = false;
this.signalRUrl = null;
this.ticketLockSignalRHubProxy = null;
this.signalRConnection = null;
}
public void doWork()
{
this.connectToSignalR();
// go ahead and do something
}
private void connectToSignalR()
{
if (this.signalRUrl == null)
{
if (!Utils.signalRWebApp.started)
{
Utils.signalRWebApp
.start(this.signalRBackplaneConnectionString());
}
this.signalRUrl = Utils.signalRWebApp.signalRUrl;
}
if (this.ticketLockSignalRHubProxy == null)
{
this.signalRConnection = new HubConnection(this.signalRUrl);
this.ticketLockSignalRHubProxy = this.signalRConnection.CreateHubProxy("TicketLockSignalRHub");
this.signalRConnection.Start().Wait();
}
}
}
And everything works, in the old Daemon framework. But in the new, I get an exception on this.signalRConnection.Start():
System.AggregateException
Message "One or more errors occurred." string
InnerException {
"StatusCode: 500,
ReasonPhrase: 'Internal Server Error',
Version: 1.1,
Content: System.Net.Http.StreamContent,
Headers:
{
Date: Mon, 30 Mar 2020 16:29:27 GMT
Server: Microsoft-HTTPAPI/2.0
Content-Length: 0
}"
}
System.Exception {
Microsoft.AspNet.SignalR.Client.HttpClientException
}
So the question is, where do I go look to find out why the SignalR server is throwing a 500?
And why would this work in the old framework and not in the new one?
The only structural difference I can see is that in the new framework the Start() method is being called from a Timer ElapsedEventHandler. Could that make a difference?
FWIW: I tried to initialize signalr tracing, in accordance with the instructions here:
Using the old framework that works, I see:
SignalR.SqlMessageBus Information: 0 : SignalR SQL objects installed SignalR.SqlMessageBus Verbose: 0 : Created DbCommand: CommandType=Text, CommandText=SELECT [PayloadId] FROM [SignalR].[Messages_0_Id], Parameters= SignalR.ScaleoutMessageBus Information: 0 : Stream(0) - Changed state from Initial to Open SignalR.SqlMessageBus Verbose: 0 : Stream 0 : SqlReceiver started, initial payload id=4188906 SignalR.SqlMessageBus Verbose: 0 : Stream 0 : Executing receive reader, initial payload ID parameter=4188906
SignalR.SqlMessageBus Verbose: 0 : Stream 0 : Starting SQL notification listener SignalR.SqlMessageBus Verbose: 0 : Stream 0 : SQL notification listener started
Using the new framework that does not:
SignalR.SqlMessageBus Information: 0 : SignalR SQL objects installed SignalR.SqlMessageBus Verbose: 0 : Created DbCommand: CommandType=Text, CommandText=SELECT [PayloadId] FROM [SignalR].[Messages_0_Id], Parameters= SignalR.ScaleoutMessageBus Information: 0 : Stream(0) - Changed state from Initial to Open SignalR.SqlMessageBus Verbose: 0 : Stream 0 : SqlReceiver started, initial payload id=4188977 SignalR.SqlMessageBus Verbose: 0 : Stream 0 : Executing receive reader, initial payload ID parameter=4188977
SignalR.SqlMessageBus Verbose: 0 : Stream 0 : Starting SQL notification listener
In other words, in the new framework, we see "Starting SQL notification listener" but we never see "SQL notification listener started".
Any ideas as to why it's not starting?
Or where I can look to discover why it's not starting?
