Opensearch distribution version 2.0

[satyavadali]

With new Opensearch distribution version 2.0 - create index fails with

elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'request [/elastalert_host_user/_doc/_mapping] contains unrecognized parameter: [include_type_name]')

opensearch-project/OpenSearch#2195

hard coding to esversion = "8.2.0" works.

thanks

[jertel]

The PR for this has been merged, and it will be included in the next release, which I hope to have out this week. #880

[mharrys]

@jertel This does still not seem to work for me despite the update. I am running "opensearchproject/opensearch:2.1.0" and "jertel/elastalert2:2.6.0". I looked in the code how the version was determined. I downloaded the same elasticsearch==7.10.1 python lib and when I dump what client.info() gave it did not include distribution field nor did it report OpenSearch version but rather the last compatible ElasticSearch version it would seem. The effect is that ElastAlert2 thinks I am running 7.X/1.X and I get the error.

{
    "name": "opensearch-cluster-master-0",
    "cluster_name": "opensearch-cluster",
    "cluster_uuid": "w7yfnKEgR2qP4fx2CxLV0w",
    "version": {
        "number": "7.10.2",
        "build_type": "tar",
        "build_hash": "388c80ad94529b1d9aad0a735c4740dce2932a32",
        "build_date": "2022-06-30T21:31:04.823801692Z",
        "build_snapshot": false,
        "lucene_version": "9.2.0",
        "minimum_wire_compatibility_version": "7.10.0",
        "minimum_index_compatibility_version": "7.0.0"
    },
    "tagline": "The OpenSearch Project: https://opensearch.org/"
}

Unless there is a setting I need to use in OpenSearch? I did not find anything though

[jertel]

See if you're running with compatibility mode enabled. It might explain this behavior you're seeing from your cluster: https://repost.aws/questions/QUwL9qwhP3QGCB7VAx2mxH8A/how-to-enable-compatibility-mode-on-existing-opensearch-domain

[mharrys]

Thanks, that appears to be it.

(Posting here in case link changes in the future) Running this

PUT /_cluster/settings
{
  "persistent" : {
    "compatibility.override_main_response_version" : false
  }
}

now gives my client.info() the following

{
    "name": "opensearch-cluster-master-0",
    "cluster_name": "opensearch-cluster",
    "cluster_uuid": "w7yfnKEgR2qP4fx2CxLV0w",
    "version": {
        "distribution": "opensearch",
        "number": "2.1.0",
        "build_type": "tar",
        "build_hash": "388c80ad94529b1d9aad0a735c4740dce2932a32",
        "build_date": "2022-06-30T21:31:04.823801692Z",
        "build_snapshot": false,
        "lucene_version": "9.2.0",
        "minimum_wire_compatibility_version": "7.10.0",
        "minimum_index_compatibility_version": "7.0.0"
    },
    "tagline": "The OpenSearch Project: https://opensearch.org/"
}

and no more error when running elastalert2.