44

I'm using ASP.NET Core 2.1. I have settings in appsettings.json and I bind them to classes using the options pattern. I want to override some of them in appsettings.Production.json.

Overriding is supported according to the docs, and works for me generally. But it doesn't work for arrays.

appsettings.json:

"MySectionOuter": {
  "MySectionInner": [
    {
      "foo": "1",
      "bar": "2",
      "baz": "3"
    },
    {
      "foo": "a",
      "bar": "b",
      "baz": "c"
    }
  ]
}

My overrides in appsettings.Production.json

"MySectionOuter": {
  "MySectionInner": [
    {
      "bar": "4",
    },
    {
      "baz": "d"
    }
  ]
}

However that doesn't work - it adds rather than replaces.

I read that the array is syntactic sugar for a key-value store. So I also tried this:

"MySectionOuter": {
  "MySection:1": {
    "bar": "4",
  },
  "MySection:2": {
    "baz": "b",
  }
}

But that also doesn't work.

What is the correct syntax?

UPDATE

The comments show I haven't explained properly. What I want is like this:

During development:

element1: foo=1
element1: bar=2
element1: baz=3
element2: foo=a
element2: bar=b
element2: baz=c

During production:

element1: foo=1
element1: bar=2
element1: baz=4  // this was changed
element2: foo=a
element2: bar=b
element2: baz=d  // this was changed
lonix
  • 14,255
  • 23
  • 85
  • 176
  • Why use an *array* at all, especially when the contents are unrelated? Apart from making it hard to override values, you can't read the contents into a List or any other kind of strongly typed container. You'd have to read individual values. – Panagiotis Kanavos Oct 11 '18 at 08:22
  • Besides, there's *no order guarantee* in configuration settings. A provider (eg a database provider) may return them in any order and configuration will (should) still work the same. The JSON hierarchy is only used to generate the key path, nothing more – Panagiotis Kanavos Oct 11 '18 at 08:24
  • 1
    Thanks for pointing that out - it was only an example, and I've changed it. – lonix Oct 11 '18 at 08:28
  • @Ionix the answer is still the same. *Everything* is a path to the values. Maxim explains how that works. Your second example overrides only `MySectionOuter::MySection:1::bar` and `MySectionOuter::MySection:2::baz`, not the other values. Putting that `1` in the name didn't change that. – Panagiotis Kanavos Oct 11 '18 at 08:33
  • @PanagiotisKanavos I'm not sure I understand you - what I want is to only override those values that I specified in the production file, and for the rest to remain the same. – lonix Oct 11 '18 at 08:36
  • @Ionix .NET Core configuration isn't JSON-specific. It can use any number of providers including ini files, XML, YAML, databases as long as the providers returns them as individual values with a key path. That path is always `Section1::Subsection1::Whatever::...::SettingName`. No matter the file's structure, all settings are reduced to that form. If you want to load a different list of data for each environment you should probably use a different mechanism – Panagiotis Kanavos Oct 11 '18 at 08:37
  • @Ionix in that case you should probably post code that reproduces the problem. The values you override are `MySectionOuter::MySection:1::bar` and `MySectionOuter::MySection:2::baz`. All others should remain the same - `MySectionOuter::MySection:1::foo` should still be `"1"` – Panagiotis Kanavos Oct 11 '18 at 08:39
  • @PanagiotisKanavos Ok so now we are on the same page. But Maxims answer covers why it is not intuituve, I ran into those exact problems. I'll have to use a different approach. Thanks. – lonix Oct 11 '18 at 08:45
  • @PanagiotisKanavos: I don't think you meant it this way, but your comment could be taken to mean that the order of config *providers* is not deterministic. It in fact is. Providers are loaded in the order they are defined. However, the order of individual settings that provider is loading may not always be the same. – Chris Pratt Oct 11 '18 at 12:55
  • @ChrisPratt the question is about arrays of settings. Different providers may return results in different order thus generating a different path. A database provider for example would have no way to decide how to order results like those posted in the question – Panagiotis Kanavos Oct 11 '18 at 12:58
  • @PanagiotisKanavos By redefining the settings in the production file, in the same order, I get the desired effect. Are you suggesting that I can't rely on that in the future? – lonix Oct 11 '18 at 13:14
  • 2
    @Ionix you can depend on the order as far as file-based providers are concerned. If you add another provider, eg a database, you'll have to ensure there's *something* that can be used to rebuild the full key. A database could return results in any order. If the database record contained an `Index` field though, you could use that to build the same setting path generated by a file. – Panagiotis Kanavos Oct 11 '18 at 13:26

3 Answers3

51

In fact, there are no arrays there when the configuration is built. It's just a key-value pair dictionary. So you end up with string keys, something like "mysectionouter:mysectioninner:0:foo" = 1.

So when in your config you define an array, the following happens:

appsettings.json:

"mysectionouter:mysectioninner:0:foo" = 1
"mysectionouter:mysectioninner:0:bar" = 2

appsettings.production.json:

"mysectionouter:mysectioninner:0:bar" = new1

result:

foo = 1
bar = new1

So it's just index-based, and next configuration just overrides a key. In your second example, you achieve nothing but changing the index. Representation would be:

"mysectionouter:mysectioninner:1:bar" = new1

So back to your question: arrays are tricky in appsettings, and though supported, are generally hard and not intuitive to use.

By index you may get a weird merge of two not related objects, if you define different sets of settings in your files, like settings A and B in the first config, and C in second, you will get C and B in the result, and you likely don't want to have B at all. Worse still, you can get a mix of A and C if you define only some fields of each object.

I'd recommend using some other files for storing this kind of information. You can also break in the debugger just where the configuration is loaded and see for yourself how these keys are build to get more insight.

Rajan Mishra
  • 1,178
  • 2
  • 14
  • 30
Maxim Zabolotskikh
  • 3,091
  • 20
  • 21
22

According to this blog post: https://www.paraesthesia.com/archive/2018/06/20/microsoft-extensions-configuration-deep-dive/

It's not possible to remove configuration items with a provider.

You can add configuration at override time, but you can’t remove things. The best you can do is override a value with an empty string.

Instead you should only fill as little information as needed in the appsettings.config and fill the appropriate settings in a more specialized settings file. E.g. appsettings.Development.config or your appsettings.Production.config. Or as suggested in the blog post:

Since you can’t remove things, specify as little configuration as possible and behave correctly using defaults in your code when configuration isn’t there.

Philipp Grathwohl
  • 2,726
  • 3
  • 27
  • 38
  • 2
    The question is about *replacing*, not removing. Replacement works just fine, as long as the *key* is the same. With arrays, the index is part of the key – Panagiotis Kanavos Oct 11 '18 at 08:21
  • 3
    @PanagiotisKanavos The question asks that the array should contain less elements after the `appsettings.production.config` was applied. Since each array element has its own key, they would have to be _removed_ from the configuration. So I think the answer still applies. – Philipp Grathwohl Oct 11 '18 at 08:24
  • The array contains the same number of elements. It's those *elements* though that have fewer properties. The OP assumed the array or the elements themselves would be replaced. Probably didn't realize that only the settings themselves matter and everything else is just the path to the values – Panagiotis Kanavos Oct 11 '18 at 08:25
  • 1
    You are right! I will leave the answer because the suggestion from the blog post still makes sense - and when applied the problem would disappear. – Philipp Grathwohl Oct 11 '18 at 08:28
  • 1
    This fixed it for me very nicely. I just leave those array setting empty in the root file. Perfect! – Matthew MacFarland Aug 12 '21 at 22:45
3

I was actually having a similar issue in dotnet 6, when trying to merge arrays from multiple appsettings, when I stumbled across this thread.
The solution was actually way simpler then thought.
Microsoft.Extensions.Configuration merges arrays through the index:
{ foo: [1, 2, 3] } + { foo: [4, 5] } = { foo: 4, 5, 3 }

But we want to be able to declare which entries override others and which ones should be added to the list. And we do this by declaring a GUID as dictionary key instead of an array.

{
  "foo": {
     "870622cb-0372-49f3-a46e-07a1bd0db769": 1,
     "cbb3af55-94ea-41a5-bbb5-cb936ac47249": 2,
     "9410fcdc-28b3-4bff-bfed-4d7286b33294": 3
  }
}
+
{
  "foo": {
    "cbb3af55-94ea-41a5-bbb5-cb936ac47249": 4,
    "1c43fa78-b8db-41f8-809d-759a4bc35ee2": 5,
  }
}
=
{
  "foo": {
    "870622cb-0372-49f3-a46e-07a1bd0db769": 1,
    "cbb3af55-94ea-41a5-bbb5-cb936ac47249": 4, /*2 changed to 4 because key equals*/
    "9410fcdc-28b3-4bff-bfed-4d7286b33294": 3
    "1c43fa78-b8db-41f8-809d-759a4bc35ee2": 5, /*while 5 added to the list*/
  }
}

This may seem inconventient at first, because one would think, that ((IConfiguration)config).GetSection("foo").Get<int[]>();

Throws some kind of invalid type exception, since a Guid is not what we know as array index. But it can actually (implicitly!) return the merged "foo" section above as int[].

Canabale
  • 157
  • 1
  • 7