Manage replicas of ISM indices with templates

Question

I have a standalone instance with Opensearch for testing purposes, I want to keep it light and clean so I'm using ISM to delete indices older than x days.

What I noticed is that by default Opensearch generates a management index (".opensearch-ism-config") with replica "1".

Since I'm using a standalone instance (it is just testing, I'm not worried with redundancy, HA or anything like that) and want to keep my cluster with green status, I have decided that I want those indices to have replica "0".

In order to achieve that, I have created a template in which I set replica "0" for these indices:

{
  "order" : 100,
  "version" : 1,
  "index_patterns" : [".opensearch-ism-*"],
  "settings" : {
    "index": {
      "number_of_shards" : "1",
      "number_of_replicas": 0
    }
  }
}

After a PUT, I start using ISM so that the management ISM index is created after this template is on Opensearch node.

What I observe is that all management indices from ISM are generated with replica "1", therefore ignoring the template.

I can set replica to "0" by updating index settings after creation but this is not the ideal scenario as ISM index rotate and new ones are generated from time to time.

Is there any way to have ISM indices applying replica "0" automatically ?

"What I observe is that all management indices from ISM are generated with replica "0", therefore ignoring the template". Do you mean "1"? 0 means all good no? Also maybe you meant ".opendistro" and not ".opensearch"? — Riz, May 16 '22 at 23:32
There is just one index called `.opendistro-ism-config`. Wat do you mean by "all"? In your case only `.opendistro-ism-config` should have 0 replica and no other index. So are you saying that '.opendistro-ism-config' index still have 1 replica after index rollover? I am not even sure if there is a roll over for this index as this index has all the config for ISM. Be very carefull with this. — Riz, May 17 '22 at 08:25
There is another index .opendistro-job-scheduler-lock also with replica "1". If these indices don't rollover I can PUT replica "0" after the index ceation. Although, I think the proper behaviour should be that these indices respect the index template. — João Pereira, May 18 '22 at 10:07
But your index pattern doesn't match ".opendistro-job-scheduler-lock"! — Riz, May 18 '22 at 12:03

score 0 · Answer 1 · answered Aug 24 '23 at 05:27

I ran into this same issue when transitioning from Elasticsearch 6 to OpenSearch. In Elasticsearch I was accustomed to index templates being additive. For example, in Elasticsearch 6, I could do something like this:

Create an index template to configure ILM rollover alias

PUT _template/a_descriptive_template_name-ilm
{
    "settings": {
        "index.lifecycle.name": "an_efficient_ilm_policy",
        "index.lifecycle.rollover_alias": "a_descriptive_index_name-ilm"
    },
    "order": 10,
    "index_patterns": [
        "a_descriptive_index_name-ilm-*"
    ]
}

Create an index template to configure index shard settings

PUT _template/a_common_template_name
{
    "settings": {
        "number_of_shards": 2,
        "number_of_replicas": 1
    },
    "order": 5,
    "index_patterns": [
        "a_descriptive_index_name-ilm-*"
    ]
}

Create the initial index

PUT a_descriptive_index_name-ilm-000001
{
    "aliases": {
        "a_descriptive_index_name-ilm": {
            "is_write_index": true
        }
    }
}

Verify the settings

GET a_descriptive_index_name-ilm-000001
{
  "a_descriptive_index_name-ilm-000001" : {
    "aliases" : {
      "a_descriptive_index_name-ilm" : {
        "is_write_index" : true
      }
    },
    "mappings" : { },
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "an_efficient_ilm_policy",
          "rollover_alias" : "a_descriptive_index_name-ilm"
        },
        "number_of_shards" : "2",
        "provided_name" : "a_descriptive_index_name-ilm-000001",
        "creation_date" : "1692853100041",
        "number_of_replicas" : "1",
        "uuid" : "AJ3OiRaXRj2cVCvuKlUWgg",
        "version" : {
          "created" : "6081299"
        }
      }
    }
  }
}

When I tried to do this same technique in Opensearch 2.3, I noticed that it did not work as I expected. The template with the higher priority overwrote the settings in the lower priority template, completely, even if there were not overlapping settings. In the example below, where I am doing the same thing as Elasticsearch (but using different verbs for OpenSearch), you will see that the additive behavior is no longer supported:

Create an index template to configure ISM rollover alias

PUT _index_template/a_descriptive_template_name-ism
{
    "template": {
        "settings": {
            "plugins.index_state_management.rollover_alias": "a_descriptive_index_name-ism"
        }
    },
    "priority": 10,
    "index_patterns": [
        "a_descriptive_index_name-ism-*"
    ]
}

Create an index template to configure index shard settings

PUT _index_template/a_common_template_name
{
    "template": {
        "settings": {
          "number_of_shards": 2,
          "number_of_replicas": 1
        }
    },
    "priority": 5,
    "index_patterns": [
        "a_descriptive_index_name-ism-*"
    ]
}

Create the initial index

PUT a_descriptive_index_name-ism-000001
{
    "aliases": {
        "a_descriptive_index_name-ism": {
            "is_write_index": true
        }
    }
}

Verify the settings

{
  "a_descriptive_index_name-ism-000001" : {
    "aliases" : {
      "a_descriptive_index_name-ism" : {
        "is_write_index" : true
      }
    },
    "mappings" : { },
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "plugins" : {
          "index_state_management" : {
            "rollover_alias" : "a_descriptive_index_name-ism"
          }
        },
        "provided_name" : "a_descriptive_index_name-ism-000001",
        "creation_date" : "1692853523944",
        "number_of_replicas" : "1",
        "uuid" : "ZZSjabAVQACC7m8Xn0uKtA",
        "version" : {
          "created" : "136247827"
        }
      }
    }
  }
}

As you can see, the "number_of_shards": 2 setting gets overwritten.

To apply both the shard settings and the ISM rollover settings, (and whatever other settings / mappings required), there are two options.

Combine all the settings / mappings and ISM configuration into a single index template (or single index template per index pattern)
[Better Solution] Use the OpenSearch Composable index templates functionality to allow for "layering" templates.

The OP answered in a later comment "It does, I have another index template". To solve for this scenario, option 2 is the way I suggest to follow.

Here is an example:

Create a component template to configure index shard settings

PUT _component_template/shards_component_template
{
      "template": {
        "settings": {
          "number_of_shards": 2,
          "number_of_replicas": 1
        }
    }
}

Create the index template to configure ILM rollover alias, referencing the component template

PUT _index_template/a_descriptive_template_name-ism
{
    "template": {
        "settings": {
            "plugins.index_state_management.rollover_alias": "a_descriptive_index_name-ism"
        }
    },
    "priority": 10,
    "index_patterns": [
        "a_descriptive_index_name-ism-*"
    ],
    "composed_of": [
        "shards_component_template"
    ]
}

Create the initial index

PUT a_descriptive_index_name-ism-000001
{
    "aliases": {
        "a_descriptive_index_name-ism": {
            "is_write_index": true
        }
    }
}

Verify the settings

{
  "a_descriptive_index_name-ism-000001" : {
    "aliases" : {
      "a_descriptive_index_name-ism" : {
        "is_write_index" : true
      }
    },
    "mappings" : { },
    "settings" : {
      "index" : {
        "number_of_shards" : "2",
        "plugins" : {
          "index_state_management" : {
            "rollover_alias" : "a_descriptive_index_name-ism"
          }
        },
        "provided_name" : "a_descriptive_index_name-ism-000001",
        "creation_date" : "1692854275856",
        "number_of_replicas" : "1",
        "uuid" : "SGvVILFsSG-3PdpoCg8Tkw",
        "version" : {
          "created" : "136247827"
        }
      }
    }
  }
}

Manage replicas of ISM indices with templates

1 Answers1