Missing required property 'NodeInfoAction.destructiveRequiresName'

[patrickmann]

When the UI initializes after login, this exception appears. It only shows up once and I don't notice any functional issues (though I can't tell if telemetry is affected).

Expected Behavior

No exception

Current Behavior

2024-03-14 08:52:24,570 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
org.graylog.shaded.opensearch2.org.opensearch.OpenSearchException: Could not read nodes list from indexer!
at org.graylog.storage.opensearch2.OpenSearchClient.exceptionFrom(OpenSearchClient.java:254) ~[classes/:?]
at org.graylog.storage.opensearch2.OpenSearchClient.execute(OpenSearchClient.java:177) ~[classes/:?]
at org.graylog.storage.opensearch2.ClusterAdapterOS2.nodesInfo(ClusterAdapterOS2.java:299) ~[classes/:?]
at org.graylog2.telemetry.rest.TelemetryService.getSearchClusterInfo(TelemetryService.java:209) ~[classes/:?]
...
Caused by: org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'NodeInfoAction.destructiveRequiresName'

Possible Solution

Per @dennisoelkers this may be due to in progress client migration

Steps to Reproduce (for bugs)

1. Restart server and login

Your Environment

• Graylog Version: 6.0.0 alpha3
• OpenSearch Version: 2.11.1

[patrickmann]

Full stack trace:

2024-03-14 09:13:31,401 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
org.graylog.shaded.opensearch2.org.opensearch.OpenSearchException: Could not read nodes list from indexer!
at org.graylog.storage.opensearch2.OpenSearchClient.exceptionFrom(OpenSearchClient.java:254) ~[classes/:?]
at org.graylog.storage.opensearch2.OpenSearchClient.execute(OpenSearchClient.java:177) ~[classes/:?]
at org.graylog.storage.opensearch2.ClusterAdapterOS2.nodesInfo(ClusterAdapterOS2.java:299) ~[classes/:?]
at org.graylog2.telemetry.rest.TelemetryService.getSearchClusterInfo(TelemetryService.java:209) ~[classes/:?]
at org.graylog2.telemetry.rest.TelemetryService.getTelemetryResponse(TelemetryService.java:114) ~[classes/:?]
at org.graylog2.telemetry.rest.TelemetryResource.get(TelemetryResource.java:69) ~[classes/:?]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) ~[jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:261) [jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [jersey-common-3.1.5.jar:?]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [jersey-common-3.1.5.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [jersey-common-3.1.5.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [jersey-common-3.1.5.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [jersey-common-3.1.5.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [jersey-common-3.1.5.jar:?]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:240) [jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:697) [jersey-server-3.1.5.jar:?]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:367) [jersey-container-grizzly2-http-3.1.5.jar:?]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:190) [grizzly-http-server-4.0.1.jar:4.0.1]
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) [metrics-core-4.2.25.jar:4.2.25]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'NodeInfoAction.destructiveRequiresName'
at org.opensearch.client.util.ApiTypeHelper.requireNonNull(ApiTypeHelper.java:89) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.opensearch.nodes.info.NodeInfoAction.(NodeInfoAction.java:57) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.opensearch.nodes.info.NodeInfoAction.(NodeInfoAction.java:50) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.opensearch.nodes.info.NodeInfoAction$Builder.build(NodeInfoAction.java:114) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.opensearch.nodes.info.NodeInfoAction$Builder.build(NodeInfoAction.java:94) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:92) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:369) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:355) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.transport.rest_client.RestClientTransport.decodeResponse(RestClientTransport.java:317) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.transport.rest_client.RestClientTransport.getHighLevelResponse(RestClientTransport.java:283) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:143) ~[opensearch-java-2.9.0.jar:?]
at org.opensearch.client.opensearch.nodes.OpenSearchNodesClient.info(OpenSearchNodesClient.java:145) ~[opensearch-java-2.9.0.jar:?]
at org.graylog.storage.opensearch2.ClusterAdapterOS2.lambda$nodesInfo$19(ClusterAdapterOS2.java:299) ~[classes/:?]
at org.graylog.storage.opensearch2.OpenSearchClient.execute(OpenSearchClient.java:175) ~[classes/:?]
... 31 more

[moesterheld]

@dennisoelkers
This is most likely caused by an incompatibility in OS versions.
I had the error in 2.10. I tried setting action.destructive_requires_name by hand, however it was never picked up by the nodes info endpoint. It went away after upgrading to 2.11

Unfortunately, it is expected to be there by the client's NodeInfoAction object

[moesterheld]

Sorry, I just saw that @patrickmann is already on 2.11 . Can you try to set the property using _cluster/settings to see if it goes away?

[patrickmann]

Can you try to set the property using _cluster/settings to see if it goes away?

@moesterheld When I add that setting I then get the other error you mentioned:

2024-03-14 13:51:38,535 ERROR: org.graylog2.periodical.IndexerClusterCheckerThread - Error while trying to check Elasticsearch disk usage.Details: Cannot invoke "jakarta.json.JsonValue.asJsonObject()" because "current" is null

[dennisoelkers]

This is related to a client bug: opensearch-project/opensearch-java#894

[moesterheld]

@dennisoelkers
I will write a note to set action.destructive_requires_name to the 6.0 release notes and add it explicitly in the data node config for OS to avoid problems until this is potentially fixed.

[janheise]

fixed by #18648