3

I am currently working on an asp.net core (c#) Web API Project.

Each call in to the Web API will have a unique key to help identify the user making the call.

Each caller will have a specific set of configuration values associated to it. I expect there will be about 100 to 200 different users who will access this Web API. I want to try and hold all the configurations for each caller in a configuration file using the unique key to separate each. I expect there will be around 30 to 50 settings per caller in the configuration file (could be up to 100 setting for some).

My question: Is using the AppSetting.json file a good approach to hold these values, or should I create a separate configuration json file for each called and load when required? Efficiency is something that will need to be considered.

user2864740
  • 60,010
  • 15
  • 145
  • 220
Junaid Vindhani
  • 31
  • 1
  • 2
    Is there some reason you are not using a database to hold user data? Configuration files are best for application configuration, e.g., per environment, and shouldn't really be used for user profile data. – Polyfun May 19 '20 at 09:55
  • Where does the application store/get other data? Also, are the combinations of settings really unique per user, or could you create a number of profiles to share between users? – Bernard Vander Beken May 19 '20 at 10:19
  • "Best" should generally be avoided in questions, as there are many _viable_ solutions, with specific pros and cons.. that said, neither of the approaches presented are what I'd recommend (or would want to maintain). It might be worth considering taking a step back from the artificially imposed either-or approach. Anyway, consider websites that have user accounts and many dozens of settings or pieces of information about a user: "most" are using SQL or some other database model. – user2864740 May 19 '20 at 10:22
  • Ihow big are the settings? 1kb-100kb per user? Just use a Dictionary you load once in the StartUp, otherwise if they're larger in size you will want a Cache, NoSQL or RDBMS solution. Good luck – Jeremy Thompson May 19 '20 at 10:46
  • Thank you for the responses. At this stage I have not considered using a Database as I didn't want the overhead. But reading the comments I think maybe I should take a step back and reconsider the solution. My take away message is that the config approach is not the most pragmatic. – Junaid Vindhani May 19 '20 at 12:37

2 Answers2

0

In a given context it looks like the configuration per client is a part of your solution. I would strongly consider moving the configuration to the code and handle it as a normal part of the business domain.

Saving such things in an appsetting.json looks dangerous (easy to make a change in a configuration for a wrong client) and hard to maintain. This may also impact debugging as you might have a different set of configuration files on a dev and prod environments.

There is also a performance consideration. Reading numerous files from the disk is not the best thing to do. You could always cache it in the memory but then your memory footprint increases.

4rchie
  • 1,244
  • 1
  • 15
  • 23
  • 1
    One point I've learnt is reading numerous files from disk for example an image file system will out perform a dB. Especially if you have a folder structure where the bottom directory has < 512 files. The dB is generally a data file, well two, the Log/Wal/Archive file with transactions committed to the dB file or partitions. You suggesting caching such as Redis or MemCacheD I think would be an excellent solution. – Jeremy Thompson May 19 '20 at 10:25
  • A standard DB will be plenty fine "performance", especially for this trivial load (buffers = awesome). The LOG/WAL/locking etc. concerns are basically non-relevant when not doing updates.. and even with updates "slowing performance" (not relevant here), using a DB is about useful information management, not caching. Of course, it's possible to write poor queries against a not-sufficient schema.. then again, it's also possible to write terrible disk access or distributed cache access code. – user2864740 May 19 '20 at 10:30
  • @user2864740 indeed, that's an important point about updates, vacuuming and immediate vs eventual consistency. – Jeremy Thompson May 19 '20 at 10:35
0

I'd be thinking SQLite DB over JSON files.

For 100 users with 50 settings it will be much easier to manage in a dB and that will prevent file locking during periods of high concurrency in multi-threaded situations. If the settings aren't relational a better choice would be a NoSQL db.

Managing 5,000 settings spread across 100 to 200 files will be a nightmare. Consider NoSQL or a lightweight RDBMS if doing bulk SELECTs, UPDATEs, (INSERTs for app upgrades) etc is necessary.

It's so small the performance differences will be barely noticeable, hence the easiest to read, maintain and build on would be my professional advice.

Jeremy Thompson
  • 61,933
  • 36
  • 195
  • 321
  • Thank you very much for the response. If I used the AppSetting for all the configuration, would I still need to give consideration to file locking? I assumed that as part of the process the middle-ware will load config values into memory. – Junaid Vindhani May 19 '20 at 12:43
  • Typically you store app wide settings in the JSON app settings and using Configuration.GetSetting won't lock it. I once did a system like this with files (to my bosses Mums delight) only to bump into massive issues, particularly security. File locking happens when there's contention, see this: https://stackoverflow.com/a/11060322/495455 – Jeremy Thompson May 19 '20 at 19:56