Error in indexing

[praveeng-nair]

Hi Team,
My conductor is using ES5 module but I am running it on AWS ES6.

"version" : {
    "number" : "6.8.0",
    "build_flavor" : "oss",
    "build_type" : "zip",
    "build_hash" : "f27534d",
    "build_date" : "2021-11-03T19:01:02.985059Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  } 

I am getting below error and I am not sure what is the fix for this.

Started server on http://localhost:8080/
{"exception":{"stacktrace":"java.lang.RuntimeException: Operation 'indexWithRetry:Bulk Indexing task' failed after retrying 3 times, retry limit 3\n\tat com.netflix.conductor.common.utils.RetryUtil.retryOnException(RetryUtil.java:127)\n\tat com.netflix.conductor.dao.es5.index.ElasticSearchRestDAOV5.indexWithRetry(ElasticSearchRestDAOV5.java:825)\n\tat com.netflix.conductor.dao.es5.index.ElasticSearchRestDAOV5.indexBulkRequest(ElasticSearchRestDAOV5.java:811)\n\tat com.netflix.conductor.dao.es5.index.ElasticSearchRestDAOV5.indexObject(ElasticSearchRestDAOV5.java:804)\n\tat com.netflix.conductor.dao.es5.index.ElasticSearchRestDAOV5.indexTask(ElasticSearchRestDAOV5.java:433)\n\tat com.netflix.conductor.core.orchestration.ExecutionDAOFacade.updateTask(ExecutionDAOFacade.java:352)\n\tat com.netflix.conductor.core.execution.WorkflowExecutor.terminate(WorkflowExecutor.java:1466)\n\tat com.netflix.conductor.core.execution.WorkflowExecutor.decide(WorkflowExecutor.java:1060)\n\tat com.netflix.conductor.core.execution.WorkflowExecutor.terminateWorkflow(WorkflowExecutor.java:756)\n\tat com.netflix.conductor.core.execution.tasks.SubWorkflow.cancel(SubWorkflow.java:166)\n\tat more","exception_class":"java.lang.RuntimeException",
"exception_message":"Operation 'indexWithRetry:Bulk Indexing task' failed after retrying 3 times, retry limit 3"},"source_host":"conductor-server-7965c76b4-75v6g","method":"indexWithRetry","level":"ERROR","message":"Failed to index org.elasticsearch.action.bulk.BulkRequest@520e0b19 for request type: task","mdc":{},"@timestamp":"2022-01-29T04:25:28.213Z","file":"ElasticSearchRestDAOV5.java","line_number":"839","thread_name":"pool-17-thread-1","@version":1,"logger_name":"com.netflix.conductor.dao.es5.index.ElasticSearchRestDAOV5","class":"com.netflix.conductor.dao.es5.index.ElasticSearchRestDAOV5"}
1
/bin/sh: line 1:     8 Killed                  java -javaagent:/newrelic/newrelic.jar -Xms4g -Xmx4g -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 -jar conductor-server-*-all.jar /app/config/conductor-config.properties /app/config/log4j.properties

Any help is appreciated. @apanicker-nflx @aravindanr

[jxu-nflx]

@praveeng-nair Have you tried to use the es-6 module instead if you are using AWS ES6?

[praveeng-nair]

@jxu-nflx Since old version of conductor is running in prod cant really upgrade conductor version and use es6 module.

[jxu-nflx]

Hello @praveeng-nair I don't think you can use es5 client against es6 server: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/client.html

The client must have the same major version (e.g. 2.x, or 5.x) as the nodes in the cluster.

You could easily change the es module to use es6 module in your property file like this
conductor.elasticsearch.version=6