Question: How can I add a rollover_alias in the index_template file?

[thomass4t]

Hello,
I tried many different syntaxes about rollover_alias but somehow, when data-prepper is creating the template in OpenSearch, the setting gets lost.

I have a simple pipeline with a sink:

  sink:
    - opensearch:
        hosts: [ "https://127.0.0.1:9200" ]
        index: "test_logs"
        template_type: "index-template"
        template_file: "/opt/data-prepper/pipelines/log.template"
        ism_policy_file: "/opt/data-prepper/pipelines/log.policy"

The template_file looks like this:

{
  "version": 3,
  "priority": 1,
  "template": {
      "settings": {
          "index": {
             "number_of_shards": "1",
             "number_of_replicas": "0",
             "plugins": {
                "index_state_management": {
                  "rollover_alias": "test_logs"
                }
             }
          }
      }
  }
}

The shards and replicas settings are applied. However, the rollover_alias always gets lost. I also tried the older syntax "opendistro.index_state_management.rollover_alias" but this setting gets lost too.
When the template is created, I can modify it without a problem and add the rollover_alias. I just can't get it work via data-preppers template_file.
Does anybody have a clue how to do it?

[JannikBrand]

For the template_file option the following is documented for the opensearch sink plugin:

The json file content should be the json value of "template" key in the json content of OpenSearch Index templates API, e.g. otel-v1-apm-span-index-template.json

So for me it looks like you would only need to specify the following in your file:

"settings": {
   "index": {
      "number_of_shards": "1",
      "number_of_replicas": "0",
      "plugins": {
         "index_state_management": {
            "rollover_alias": "test_logs"
         }
      }
   }
}

I did not try it out, but this is how I would expect it to work

[HoffmannTom]

Thanks for your reply!
In the meantime, I chose another path and don't use template_file anymore.
This answer may help somebody else, who stumbles upon this. 👍