Elasticsearch snapshot restoring got conflicts with the internal system indices

Question

We are trying to restore a snapshot made previously, using the following command:

POST /_snapshot/my_backup/my_snapshot_2023.06.30/_restore

However, the command keeps getting errors saying ... index [.xxxxxx] because an open index with same name already exists in the cluster. .... The index name .xxxxxx indicates an internal system index.

{
  "error": {
    "root_cause": [
      {
        "type": "snapshot_restore_exception",
        "reason": "[my_backup:my_snapshot_2023.06.30/Pl0QeAojSgq1p6PYI_JdDg] cannot restore index [.ds-ilm-history-5-2023.02.17-000001] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"
      }
    ],
    "type": "snapshot_restore_exception",
    "reason": "[my_backup:my_snapshot_2023.06.30/Pl0QeAojSgq1p6PYI_JdDg] cannot restore index [.ds-ilm-history-5-2023.02.17-000001] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"
  },
  "status": 500
}

Moreover, we attempted to delete all indices by the command DELETE /_all following this post, but it does not work either. The error message is:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Wildcard expressions or all indices are not allowed"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Wildcard expressions or all indices are not allowed"
  },
  "status": 400
}

The Elasticsearch instance under the test is running on a local virtual machine, and we have backups of the virtual machine. So, we can try any command, if necessary, without risk.

Our Question:

• The command GET /_cat/indices does not show any of the internal indices. So, how to list all the internal system indices?

• We are new to the backup and restore function, so we wonder if there was anything wrong when taking the snapshot before. We used the command PUT /_snapshot/my_backup/%3Cmy_snapshot_%7Bnow%2Fd%7D%3E, and not sure if we should include additional arguments to ignore the system indices.

• Is it possible to delete all the indices, including internal and external?

We will highly appreciate any hints and suggestions.

Answer 1

Please find below my answer for each question. you have not mentioned that which version you are using hence below answer are based on latest elasticsearch version.

You can use below command to list all the indices:

GET _cat/indices/_all?expand_wildcards=all

If you want to restore the specific index then there is two option to restore it.

First option, Delete the existing specific index and restore it.

 - You can use below command to delete index or data stream.

#Delete an index
DELETE my-index

#Delete a data stream
DELETE _data_stream/logs-my_app-default

 - You can use below command to restore the specific index.

POST _snapshot/my_repository/my_snapshot_2099.05.06/_restore
{
  "indices": "my-index,logs-my_app-default"
}

Second option, Restore to rename index and reindex to original index

 - Restore to rename index (new index) first using below command:

POST _snapshot/my_repository/my_snapshot_2099.05.06/_restore
{
  "indices": "my-index,logs-my_app-default",
  "rename_pattern": "(.+)",
  "rename_replacement": "restored-$1"
}

 - Delete the original Index and reindex data from rename index to
   original index

# Delete the original index
DELETE my-index

# Reindex the restored index to rename it
POST _reindex
{
  "source": {
    "index": "restored-my-index"
  },
  "dest": {
    "index": "my-index"
  }
}

You can delete all the index and data strem using below command:

# delete all index
DELETE *?expand_wildcards=all

# delete all data stream
DELETE _data_stream/*?expand_wildcards=all

Please check this documentation for more details.

Please check this documentation for how to restore entire cluster.