0

I'm trying to figure out why my webservice is so slow and find ways to get it to respond faster. Current average response time without custom processing involved (i.e. apicontroller action returning a very simple object) is about 75ms.

The setup

Machine:
  • 32GB RAM, SSD disk, 4 x 2.7Ghz CPU's, 8 logical processors, x64 Windows 10
Software:
  • 1 asp.net mvc website running .net 4.0 on IISEXPRESS (System.Web.Mvc v5.2.7.0)
  • 1 asp.net web api website running .net 4.0 on IISEXPRESS (System.Net.Http v4.2.0.0)
  • 1 RabbitMQ messagebus

Asp.net Web API Code (Api Controller Action)

[Route("Send")]
[HttpPost]
[AllowAnonymous)
public PrimitiveTypeWrapper<long> Send(WebsiteNotificationMessageDTO notification)
{
    _messageBus.Publish<IWebsiteNotificationCreated>(new { Notification = notification });
    return new PrimitiveTypeWrapper<long>(1);
}

The body of this method takes 2ms. Stackify tells me there's a lot of overhead on the AuthenticationFilterResult.ExecuteAsync method but since it's an asp.net thing I don't think it can be optimized much.

Asp.net MVC Code (MVC Controller Action)

The RestClient implementation is shown below. The HttpClientFactory returns a new HttpClient instance with the necessary headers and basepath.

public async Task<long> Send(WebsiteNotificationMessageDTO notification)
{
    var result = await _httpClientFactory.Default.PostAndReturnAsync<WebsiteNotificationMessageDTO, PrimitiveTypeWrapper<long>>("/api/WebsiteNotification/Send", notification);
    if (result.Succeeded)
        return result.Data.Value;

    return 0;
}

Executing 100 requests as fast as possible on the backend rest service:

[HttpPost]
public async Task SendHundredNotificationsToMqtt()
{
    var sw = new Stopwatch();
    sw.Start();
    for (int i = 0; i < 100; i++)
    {
        await _notificationsRestClient.Send(new WebsiteNotificationMessageDTO()
        {
            Severity = WebsiteNotificationSeverity.Informational,
            Message = "Test notification " + i,
            Title = "Test notification " + i,
            UserId = 1
        });
    }
    sw.Stop();
    Debug.WriteLine("100 messages sent, took {0} ms", sw.ElapsedMilliseconds);
}

This takes on average 7.5 seconds.

Things I've tried

  1. Checked the number of available threads on both the REST service and the MVC website:

    int workers; int completions; System.Threading.ThreadPool.GetMaxThreads(out workers, out completions);

which returned for both:

Workers: 8191
Completions: 1000

  1. Removed all RabbitMQ messagebus connectivity to ensure it's not the culprit. I've also removed the messagebus publish method from the rest method _messageBus.Publish<IWebsiteNotificationCreated>(new { Notification = notification }); So all it does is return 1 inside a wrapping object.

  2. The backend rest is using identity framework with bearer token authentication and to eliminate most of it I've also tried marking the controller action on the rest service as AllowAnonymous.

  3. Ran the project in Release mode: No change

  4. Ran the sample 100 requests twice to exclude service initialization cost: No change

After all these attempts, the problem remains, it will still take about +- 75ms per request. Is this as low as it goes?

Here's a stackify log for the backend with the above changes applied. stackify performance log

The web service remains slow, is this as fast as it can get without an expensive hardware upgrade or is there something else I can look into to figure out what's making my web service this slow?

Peter
  • 14,221
  • 15
  • 70
  • 110
  • Just a couple of notes which might help you save a couple of precious milliseconds: 1. Do you reuse `HttpClient` created in your http client factory? 2. `AllowAnonymousAttribute` does not necessarily make requests bypass authentication. They will still be challenged and validated if they provide an authentication token or similar. You might want to test with authentication disabled completely. – Imantas Feb 11 '19 at 10:29
  • Thanks for the pointers @Imantas. I do not reuse the HttpClients, they get recreated for each request, though I doubt it will save >2ms. Your second tip is a good thing to investigate, thanks! – Peter Feb 11 '19 at 10:40
  • Another potential performance trap is logging, because it's setup once and then forgotten until a problem arises. Do you have a logging mechanism in your pipeline that writes every request to disc? I've seen antivirus software slowing down the whole process because it scans the logfiles on every change. – Florian Lim Feb 11 '19 at 13:50
  • 1
    you should definitely not create HttpClient on each request. go through this : https://stackoverflow.com/questions/22560971/what-is-the-overhead-of-creating-a-new-httpclient-per-call-in-a-webapi-client – Andrei Dragotoniu Feb 11 '19 at 15:07

0 Answers0