[FEATURE] Enable Generic HTTP Actions in Java Client (#910)

* [FEATURE] Enable Generic HTTP Actions in Java Client Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]> * Address code review comments Signed-off-by: Andriy Redko <[email protected]> * Address code review comments and add documentation Signed-off-by: Andriy Redko <[email protected]> * Address code review comments Signed-off-by: Andriy Redko <[email protected]> * Address code review comments Signed-off-by: Andriy Redko <[email protected]> * Address code review comments Signed-off-by: Andriy Redko <[email protected]> --------- Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Andriy Redko <[email protected]>
opensearch-project · Apr 6, 2024 · c1dcc44 · c1dcc44
1 parent 0db2215
commit c1dcc44
Show file tree

Hide file tree

Showing 24 changed files with 1,461 additions and 22 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -35,6 +35,7 @@ This section is for maintaining a changelog for all breaking changes for the cli
 
 ### Added
 - Add missed fields to MultisearchBody: seqNoPrimaryTerm, storedFields, explain, fields, indicesBoost ([#914](https://github.com/opensearch-project/opensearch-java/pull/914))
+- Add OpenSearchGenericClient with support for raw HTTP request/responses ([#910](https://github.com/opensearch-project/opensearch-java/pull/910))
 - Add missed fields to MultisearchBody: collapse, version, timeout ([#916](https://github.com/opensearch-project/opensearch-java/pull/916)
 - Add missed fields to MultisearchBody: ext, rescore and to SearchRequest: ext ([#918](https://github.com/opensearch-project/opensearch-java/pull/918)
 
@@ -362,4 +363,4 @@ This section is for maintaining a changelog for all breaking changes for the cli
 [2.5.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.4.0...v2.5.0
 [2.4.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.3.0...v2.4.0
 [2.3.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.2.0...v2.3.0
-[2.2.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.1.0...v2.2.0
+[2.2.0]: https://github.com/opensearch-project/opensearch-java/compare/v2.1.0...v2.2.0
diff --git a/USER_GUIDE.md b/USER_GUIDE.md
@@ -198,6 +198,7 @@ You can find a working sample of the above code in [IndexingBasics.java](./sampl
 - [Data Stream APIs](./guides/data_stream.md)
 - [Point-in-Time APIs](./guides/point_in_time.md)
 - [Search](./guides/search.md)
+- [Generic Client](./guides/generic.md)
 
 ## Plugins
 

diff --git a/guides/generic.md b/guides/generic.md
@@ -0,0 +1,127 @@
+- [Generic Client](#generic-client)
+  - [Getting the Client](#get-client)
+  - [Sending Simple Requests](#request-bodyless)
+  - [Sending JSON Requests](#request-json)
+  - [Sending JSON Requests using POJOs](#request-pojo)
+  - [Sending Requests using structured JSON](#request-structured)
+
+# Generic Client
+
+There are rare circumstances when the typed OpenSearch client APIs are too constraining and there is a need to communicate with OpenSearch cluster (or individual nodes) over generic HTTP request / response communication. Use `OpenSearchGenericClient` in such cases.
+
+## Getting the Client
+The following sample code gets the `OpenSearchGenericClient` from the `OpenSearchClient` instance.
+
+```java
+final OpenSearchGenericClient generic = javaClient().generic();
+```
+
+## Sending Simple Request
+The following sample code sends a simple request that does not require any payload to be provided (typically, `GET` requests).
+
+```java
+// compare with what the low level client outputs
+try (Response response = javaClient().generic().execute(Requests.builder().endpoint("/").method("GET").build())) {
+   // ...
+}
+```
+
+Please notice that the `Response` instance should be closed explicitly in order to free up any allocated resource (like response input streams or buffers), the [`try-with-resource`](https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html) pattern is encouraged.
+
+```java
+try (Response response = javaClient().generic().execute(...)) {
+   // ...
+}
+```
+
+The generic client never interprets status codes and provides the direct access to the response as it was received from the server. This is responsibility of the caller to decide what should happen in case of unsuccessful invocations.
+
+```java
+// compare with what the low level client outputs
+try (Response response = javaClient().generic().execute(...)) {
+   if (response.getStatus() != 200) {
+      // Request was not successful
+   }
+}
+```
+
+## Sending JSON Requests
+The following sample code a simple request with JSON body.
+
+```java
+ try (Response response = javaClient().generic()
+        .execute(
+            Requests.builder()
+                .endpoint("/" + index + "/_search")
+                .method("POST")
+                .json("{"
+                    + "    \"query\": {"
+                    + "        \"match_all\": {}"
+                    + "    }"
+                    + "}"
+                )
+                .build())) {
+   // Retrieve the response body as a simple string
+   final String body = response.getBody().map(Body::getAsString).orElse("");
+   // ...
+}
+```
+
+## Sending JSON Requests using POJOs
+Besides providing the ability to deal with raw request and response payloads (bodies), the `OpenSearchGenericClient` could be used mixed with existing OpenSearch typed requests and responses (POJOs), like the following sample code demonstrates.
+
+
+```java
+final JsonpMapper jsonpMapper = javaClient()._transport().jsonpMapper();
+
+final CreateIndexRequest request = CreateIndexRequest.of(
+  b -> b.index(index)
+    .mappings(
+        m -> m.properties("name", Property.of(p -> p.keyword(v -> v.docValues(true))))
+            .properties("size", Property.of(p -> p.keyword(v -> v.docValues(true))))
+    )
+    .settings(settings -> settings.sort(s -> s.field("name").order(SegmentSortOrder.Asc)))
+);
+
+try (Response response = javaClient().generic()
+      .execute(
+          Requests.builder()
+            .endpoint("/" + index).method("PUT")
+            .json(request, jsonpMapper).build())) {
+  // Retrieve the response body as a POJO
+  final CreateIndexResponse r = response.getBody()
+      .map(b -> Bodies.json(b, CreateIndexResponse._DESERIALIZER, jsonpMapper))
+      .orElse(null);
+  // ...
+}
+```
+
+## Sending Requests using structured JSON
+Dealing with strings or POJOs could be daunting sometimes, using structured JSON APIs is a middle ground of both approaches, as per following sample code that uses (`jakarta.json.Json`)[https://jakarta.ee/specifications/jsonp].
+
+```java
+try (Response response = javaClient().generic()
+      .execute(
+        Requests.builder()
+          .endpoint("/" + index)
+          .method("PUT")
+          .json(Json.createObjectBuilder()
+              .add("settings", Json.createObjectBuilder()
+                .add("index", Json.createObjectBuilder()
+                  .add("sort.field", "name"))
+                  .add("sort.order", "asc")
+              )
+              .add("mappings",Json.createObjectBuilder()
+                .add("properties", Json.createObjectBuilder()
+                  .add("name", Json.createObjectBuilder()
+                    .add("type", "keyword"))
+                    .add("doc_values", true)
+                  .add("size", Json.createObjectBuilder()
+                    .add("type", "keyword"))
+                    .add("doc_values", true))
+              )
+          )
+          .build())) {
+  // ...
+}
+```
diff --git a/java-client/src/main/java/org/opensearch/client/opensearch/OpenSearchClient.java b/java-client/src/main/java/org/opensearch/client/opensearch/OpenSearchClient.java
@@ -123,6 +123,7 @@
 import org.opensearch.client.opensearch.core.pit.ListAllPitResponse;
 import org.opensearch.client.opensearch.dangling_indices.OpenSearchDanglingIndicesClient;
 import org.opensearch.client.opensearch.features.OpenSearchFeaturesClient;
+import org.opensearch.client.opensearch.generic.OpenSearchGenericClient;
 import org.opensearch.client.opensearch.indices.OpenSearchIndicesClient;
 import org.opensearch.client.opensearch.ingest.OpenSearchIngestClient;
 import org.opensearch.client.opensearch.nodes.OpenSearchNodesClient;
@@ -155,6 +156,9 @@ public OpenSearchClient withTransportOptions(@Nullable TransportOptions transpor
     }
 
     // ----- Child clients
+    public OpenSearchGenericClient generic() {
+        return new OpenSearchGenericClient(this.transport, this.transportOptions);
+    }
 
     public OpenSearchCatClient cat() {
         return new OpenSearchCatClient(this.transport, this.transportOptions);

diff --git a/java-client/src/main/java/org/opensearch/client/opensearch/generic/Bodies.java b/java-client/src/main/java/org/opensearch/client/opensearch/generic/Bodies.java
@@ -0,0 +1,58 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.client.opensearch.generic;
+
+import jakarta.json.JsonObject;
+import jakarta.json.JsonObjectBuilder;
+import jakarta.json.stream.JsonGenerator;
+import jakarta.json.stream.JsonParser;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import org.opensearch.client.json.JsonpDeserializer;
+import org.opensearch.client.json.JsonpMapper;
+
+public final class Bodies {
+    private static final String APPLICATION_JSON = "application/json; charset=UTF-8";
+
+    private Bodies() {}
+
+    public static <C> C json(Body body, JsonpDeserializer<C> deserializer, JsonpMapper jsonpMapper) {
+        try (JsonParser parser = jsonpMapper.jsonProvider().createParser(body.body())) {
+            return deserializer.deserialize(parser, jsonpMapper);
+        }
+    }
+
+    public static <C> C json(Body body, Class<C> clazz, JsonpMapper jsonpMapper) {
+        try (JsonParser parser = jsonpMapper.jsonProvider().createParser(body.body())) {
+            return jsonpMapper.deserialize(parser, clazz);
+        }
+    }
+
+    public static <C> Body json(C value, JsonpMapper jsonpMapper) throws IOException {
+        try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
+            try (JsonGenerator generator = jsonpMapper.jsonProvider().createGenerator(baos)) {
+                jsonpMapper.serialize(value, generator);
+            }
+            return Body.from(baos.toByteArray(), APPLICATION_JSON);
+        }
+    }
+
+    public static Body json(final JsonObjectBuilder builder) {
+        return json(builder.build());
+    }
+
+    public static Body json(final JsonObject json) {
+        return json(json.toString());
+    }
+
+    public static Body json(String str) {
+        return Body.from(str.getBytes(StandardCharsets.UTF_8), APPLICATION_JSON);
+    }
+}
diff --git a/java-client/src/main/java/org/opensearch/client/opensearch/generic/Body.java b/java-client/src/main/java/org/opensearch/client/opensearch/generic/Body.java
@@ -0,0 +1,90 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.client.opensearch.generic;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.UncheckedIOException;
+import java.nio.charset.StandardCharsets;
+import javax.annotation.Nullable;
+
+/**
+ * Generic HTTP request / response body. It is responsibility of the caller to close the body instance
+ * explicitly (or through {@link GenericResponse} instance) to release all associated resources.
+ */
+public interface Body extends AutoCloseable {
+    final int DEFAULT_BUFFER_SIZE = 8192;
+
+    /**
+     * Constructs the generic response body out of {@link InputStream} with assumed content type
+     * @param body response body stream
+     * @param contentType content type
+     * @return generic response body instance
+     */
+    static @Nullable Body from(@Nullable final InputStream body, @Nullable final String contentType) {
+        if (body == null) {
+            return null;
+        } else {
+            return new GenericInputStreamBody(body, contentType);
+        }
+    }
+
+    /**
+     * Constructs the generic response body out of {@link InputStream} with assumed content type
+     * @param body response body stream
+     * @param contentType content type
+     * @return generic response body instance
+     */
+    static @Nullable Body from(@Nullable final byte[] body, @Nullable final String contentType) {
+        if (body == null) {
+            return null;
+        } else {
+            return new GenericByteArrayBody(body, contentType);
+        }
+    }
+
+    /**
+     * Content type of this body
+     * @return content type
+     */
+    String contentType();
+
+    /**
+     * Gets the body as {@link InputStream}
+     * @return body as {@link InputStream}
+     */
+    InputStream body();
+
+    /**
+     * Gets the body as {@link String}
+     * @return body as {@link String}
+     */
+    default String bodyAsString() {
+        try (final ByteArrayOutputStream out = new ByteArrayOutputStream()) {
+            try (final InputStream in = body()) {
+                final byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
+                int read;
+                while ((read = in.read(buffer, 0, DEFAULT_BUFFER_SIZE)) >= 0) {
+                    out.write(buffer, 0, read);
+                }
+            }
+
+            out.flush();
+            return new String(out.toByteArray(), StandardCharsets.UTF_8);
+        } catch (final IOException ex) {
+            throw new UncheckedIOException(ex);
+        }
+    }
+
+    /**
+     * Releases all resources associated with this body stream.
+     */
+    void close() throws IOException;
+}
diff --git a/java-client/src/main/java/org/opensearch/client/opensearch/generic/GenericByteArrayBody.java b/java-client/src/main/java/org/opensearch/client/opensearch/generic/GenericByteArrayBody.java
@@ -0,0 +1,40 @@
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * The OpenSearch Contributors require contributions made to
+ * this file be licensed under the Apache-2.0 license or a
+ * compatible open source license.
+ */
+
+package org.opensearch.client.opensearch.generic;
+
+import java.io.ByteArrayInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import javax.annotation.Nullable;
+
+/**
+ * The HTTP request / response body that uses {@link byte[]}
+ */
+final class GenericByteArrayBody implements Body {
+    private final byte[] bytes;
+    private final String contentType;
+
+    GenericByteArrayBody(final byte[] bytes, @Nullable final String contentType) {
+        this.bytes = bytes;
+        this.contentType = contentType;
+    }
+
+    @Override
+    public String contentType() {
+        return contentType;
+    }
+
+    @Override
+    public InputStream body() {
+        return new ByteArrayInputStream(bytes);
+    }
+
+    @Override
+    public void close() throws IOException {}
+}