Asp.Net Core - The configured user limit (128) on the number of inotify instances has been reached

Question

the asp.net core MVC report error after certain times of query to the mysql database (on Ubuntu 14.04/16.04) with the following message: "The configured user limit (128) on the number of inotify instances has been reached." It can be identified that the error was raised because of the controller opened too many files and exceeded the limits of iNotify setting (in the /proc/sys/fs/inotify/max_user_instances). But I was just baffled when the ASP.NET opened files on each http request and why it doesn't close the file properly? Any one encounted this issue too? Remarks: I was using Mysql.data.core and mysql.data.entityframeworkcore provider.

    private static string classiferstring = "sports,outdoor,startup,pets,child,adult,elderly";

    [AllowAnonymous]
    [HttpGet]
    public async Task<object> Classify([FromQuery] string classifyword)
    {
        string[] classifers = classiferstring.Split(',');
        if (!classifers.Contains(classifyword))
        {
            return new
            {
                status = 0,
                info = "WrongClassifier",
                Data = ""
            };
        }

        try
        {
            var predata = await (from d in _context.descriptor
                              join a in _context.combination on d.ID equals a.ID
                              select new ProductsVM
                              {
                                  CREATETIME = a.CREATETIME,
                                  ID = a.ID,
                                  COMPANY = a.COMPANY,
                                  NAME = a.NAME,
                                  PRICE = a.PRICE,
                                  TYPE = a.TYPE,
                                  HEADPHOTO = a.HEADPHOTO,
                                  REMARK = a.REMARK,

                                  Tags = d.Tags,
                                  Classifier = d.Classifier,
                                  OriginName = d.OriginName,
                                  Briefing = d.Briefing
                              }).ToListAsync();
            var data = (from x in predata
                          where x.Classifier.Contains(classifyword.ToLower())
                          select x).ToList();

            if(predata.Count<=0)
            {
                return new
                {
                    status = 2,
                    info = "NoResult",
                    Data = ""
                };
            }else
            {
                return new
                {
                    status = 1,
                    info = "Success",
                    Data = data
                };
            };
        }
        catch(Exception e)
        {
            return new
            {
                status = 0,
                info = "Error",
                Data = e.Message
            };
        }
    }

Please notice the exception was raised only in the try/catch code block instead of immediately after the action is invokded.

Many thanks if anyone has some clue to resolve this problem.

Answer 1

The best solution I found so far is to increase the fs.inotify.max_user_instances in /etc/sysctl.conf by running this command:

echo fs.inotify.max_user_instances=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

Source: https://github.com/dotnet/aspnetcore/issues/8449#issuecomment-512275929

Answer 2

For containerized environments (where config won't change for a lifetime of a container) or on build servers, the way to mitigate the issue is to set environment variable

DOTNET_HOSTBUILDER__RELOADCONFIGONCHANGE=false

This allows to continue using Host.CreateDefaultBuilder and prevents the service from creating unnecessary inotify instanced.

Answer 3

The reason why error happens is reloadOnChange cause the issue while accessing appSetting.json files.

the configured user limit (128) on the number of inotify instances has been reached

Set reloadOnChange to false:

.AddJsonFile($"appsettings.json", optional: true, reloadOnChange: false);

NOTE: if you also use the default WebHost.CreateDefaultBuilder, be aware that inside it reloadOnChange is also set to true.

So probably would be more safe to just configure your host from scratch - this is an example how to do that (a copy of Microsoft WebHost.CreateDefaultBuilder but without FileWatcher dependency and IISIntegration as you don't need IIS on Ubuntu).

Answer 4

@dmitry-pavlov is wrong, you can use the default builder as the last builder added wins - but doing so makes appsettings.json settings override all other config sources, env and command line. See https://github.com/dotnet/AspNetCore.Docs/pull/22391 where this will soon be officially documented. If you don't care about appsettings.json reloads, set it to false

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureAppConfiguration((hostingContext, config) =>
        {
            config.AddJsonFile("appsettings.json",
                optional: true,
                reloadOnChange: false);
        })
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });
}

Answer 5

I just had to close some visual studio code windows and especially files open in the visual studio code editor.

Answer 6

I recently saw this problem on dot.net 5.0. In my case it was caused by using the bundle tag helper found here: https://github.com/meziantou/Meziantou.AspNetCore.BundleTagHelpers . It seems that tag helpers which use asp-append-version have been known to cause this issue, see here https://github.com/dotnet/runtime/issues/27272#issuecomment-525007303 .