Add docs for reindex data stream REST endpoints (#120653) (#121121)

Add documentation for new REST endpoints related to data stream upgrade. 
Endpoints:
- /_migration/reindex
- /_migration/reindex/{index}/_status
- /_migration/reindex/{index}/_cancel
- /_create_from/{source}/{dest}

docs/reference/migration/apis/create-index-from-source.asciidoc

@@ -0,0 +1,142 @@
+[[indices-create-index-from-source]]
+=== Create index from source API
+++++
+<titleabbrev>Create index from source</titleabbrev>
+++++
+
+.New API reference
+[sidebar]
+--
+For the most up-to-date API details, refer to {api-es}/group/endpoint-indices[Index APIs].
+--
+
+[[indices-create-index-from-source-api-request]]
+==== {api-request-title}
+
+`PUT /_create_from/<source>/<dest>`
+
+`POST/_create_from/<source>/<dest>`
+
+[[indices-create-index-from-source-api-prereqs]]
+==== {api-prereq-title}
+
+* If the {es} {security-features} are enabled, you must have the `manage`
+<<privileges-list-indices,index privilege>> for the index.
+
+[[indices-create-index-from-source-api-desc]]
+==== {api-description-title}
+This api allows you to add a new index to an {es} cluster, using an existing source index as a basis for the new index.
+The settings and mappings from the source index will copied over to the destination index.  You can also provide
+override settings and mappings which will be combined with the source settings and mappings when creating the
+destination index.
+
+[[indices-create-index-from-source-api-path-params]]
+==== {api-path-parms-title}
+
+`<source>`::
+(Required, string) Name of the existing source index which will be used as a basis.
+
+`<dest>`::
+(Required, string) Name of the destination index which will be created.
+
+
+[role="child_attributes"]
+[[indices-create-index-from-source-api-request-body]]
+==== {api-request-body-title}
+
+`settings_override`::
+(Optional, <<index-modules-settings,index setting object>>) Settings which override the source settings.
+
+`mappings_override`::
+(Optional, <<mapping,mapping object>>) Mappings which override the source mappings.
+
+`remove_index_blocks`::
+(Optional, boolean) Filter out any index blocks from the source index when creating the destination index.
+Defaults to `true`.
+
+[[indices-create-index-from-source-api-example]]
+==== {api-examples-title}
+
+Start by creating a source index that we'll copy using this API.
+
+[source,console]
+--------------------------------------------------
+PUT /my-index
+{
+  "settings": {
+    "index": {
+      "number_of_shards": 3,
+      "blocks.write": true
+    }
+  },
+  "mappings": {
+    "properties": {
+        "field1": { "type": "text" }
+    }
+  }
+}
+--------------------------------------------------
+// TESTSETUP
+
+Now we create a destination index from the source index. This new index will have the same mappings and settings
+as the source index.
+
+[source,console]
+--------------------------------------------------
+POST _create_from/my-index/my-new-index
+--------------------------------------------------
+
+
+Alternatively, we could override some of the source's settings and mappings. This will use the source settings
+and mappings as a basis and combine these with the overrides to create the destination settings and mappings.
+
+[source,console]
+--------------------------------------------------
+POST _create_from/my-index/my-new-index
+{
+  "settings_override": {
+    "index": {
+      "number_of_shards": 5
+    }
+  },
+  "mappings_override": {
+    "properties": {
+        "field2": { "type": "boolean" }
+    }
+  }
+}
+--------------------------------------------------
+
+Since the destination index is empty, we very likely will want to write into the index after creation.
+This would not be possible if the source index contains an <<index-block-settings,index write block>> which is copied over to the destination index.
+One way to handle this is to remove the index write block using a settings override. For example, the following
+settings override removes all index blocks.
+
+
+[source,console]
+--------------------------------------------------
+POST _create_from/my-index/my-new-index
+{
+  "settings_override": {
+    "index": {
+      "blocks.write": null,
+      "blocks.read": null,
+      "blocks.read_only": null,
+      "blocks.read_only_allow_delete": null,
+      "blocks.metadata": null
+    }
+  }
+}
+--------------------------------------------------
+
+Since this is a common scenario, index blocks are actually removed by default. This is controlled with the parameter
+`remove_index_blocks`, which defaults to `true`. If we want the destination index to contains the index blocks from
+the source index, we can do the following:
+
+[source,console]
+--------------------------------------------------
+POST _create_from/my-index/my-new-index
+{
+  "remove_index_blocks": false
+}
+--------------------------------------------------

docs/reference/migration/apis/data-stream-reindex-cancel.asciidoc

@@ -0,0 +1,64 @@
+[role="xpack"]
+[[data-stream-reindex-cancel-api]]
+=== Reindex data stream cancel API
+++++
+<titleabbrev>Reindex data stream cancel</titleabbrev>
+++++
+
+.New API reference
+[sidebar]
+--
+For the most up-to-date API details, refer to {api-es}/group/endpoint-migration[Migration APIs].
+--
+
+include::{es-ref-dir}/migration/apis/shared-migration-apis-tip.asciidoc[]
+
+Cancels a running data stream reindex task which was started by the <<data-stream-reindex-api, data stream reindex API>>.
+Any backing indices that have already been reindexed and swapped into the data stream will remain in the data stream.
+Only backing indices which are currently being reindexed, or pending backing indices which are still waiting to be reindexed, will be cancelled.
+Once a data stream reindex task is cancelled it will no longer be accessible through the
+<<data-stream-reindex-status-api,status API>>. If a reindex task is not currently running
+this API will return `resource_not_found_exception`.
+
+
+///////////////////////////////////////////////////////////
+[source,console]
+------------------------------------------------------
+POST _migration/reindex
+{
+    "source": {
+        "index": "my-data-stream"
+    },
+    "mode": "upgrade"
+}
+------------------------------------------------------
+// TESTSETUP
+// TEST[setup:my_data_stream]
+///////////////////////////////////////////////////////////
+
+
+[source,console]
+----
+POST _migration/reindex/my-data-stream/_cancel
+----
+// TEST[teardown:data_stream_cleanup]
+
+[[data-stream-reindex-cancel-request]]
+==== {api-request-title}
+
+`GET /_migration/reindex/<data-stream>/_cancel`
+
+
+[[data-stream-reindex-cancel-prereqs]]
+==== {api-prereq-title}
+
+* If the {es} {security-features} are enabled, you must have the `manage`
+<<privileges-list-indices,index privilege>> for the data stream.
+
+[[data-stream-reindex-cancel-path-params]]
+==== {api-path-parms-title}
+
+`<data-stream>`::
+(Required, string)
+Name of data stream to cancel reindexing.
+

docs/reference/migration/apis/data-stream-reindex-status.asciidoc

@@ -0,0 +1,157 @@
+[role="xpack"]
+[[data-stream-reindex-status-api]]
+=== Reindex data stream status API
+++++
+<titleabbrev>Reindex data stream status</titleabbrev>
+++++
+
+.New API reference
+[sidebar]
+--
+For the most up-to-date API details, refer to {api-es}/group/endpoint-migration[Migration APIs].
+--
+
+include::{es-ref-dir}/migration/apis/shared-migration-apis-tip.asciidoc[]
+
+Obtains the current status of a reindex task for the requested data stream. This status is
+available while the reindex task is running and for 24 hours after completion of the task,
+whether it succeeds or fails. If the task is cancelled, the status is no longer available.
+If the task fails, the exception will be listed within the status.
+
+///////////////////////////////////////////////////////////
+[source,console]
+------------------------------------------------------
+POST _migration/reindex
+{
+    "source": {
+        "index": "my-data-stream"
+    },
+    "mode": "upgrade"
+}
+------------------------------------------------------
+// TESTSETUP
+// TEST[setup:my_data_stream]
+
+[source,console]
+------------------------------------------------------
+POST /_migration/reindex/my-data-stream/_cancel
+DELETE _data_stream/my-data-stream
+DELETE _index_template/my-data-stream-template
+------------------------------------------------------
+// TEARDOWN
+///////////////////////////////////////////////////////////
+
+
+[[data-stream-reindex-status-api-request]]
+==== {api-request-title}
+
+`GET /_migration/reindex/<data-stream>/_status`
+
+
+[[data-stream-reindex-status-prereqs]]
+==== {api-prereq-title}
+
+* If the {es} {security-features} are enabled, you must have the `manage`
+<<privileges-list-indices,index privilege>> for the data stream.
+
+[[data-stream-reindex-status-path-params]]
+==== {api-path-parms-title}
+
+`<data-stream>`::
+(Required, string)
+Name of data stream to get status for. The reindex task for the
+data stream should be currently running or have been completed in the last 24 hours.
+
+
+[role="child_attributes"]
+[[data-stream-reindex-status-response-body]]
+==== {api-response-body-title}
+
+`start_time`::
+(Optional, <<time-units,time value>>) The time when the reindex task started.
+
+`start_time_millis`::
+(integer) The time when the reindex task started, in milliseconds since the epoch.
+
+`complete`::
+(boolean) `false` if the reindex task is still running, and `true` if the task has completed with success or failure.
+
+`total_indices_in_data_stream`::
+(integer) The total number of backing indices in the data stream, including the write index.
+
+`total_indices_requiring_upgrade`::
+(integer) The number of backing indices that need to be upgraded. These will consist of the indices which have an
+older version and are not read-only.
+
+`successes`::
+(integer) The number of backing indices which have already been successfully upgraded.
+
+`in_progress`::
+(array of objects) Information on the backing indices which are currently being reindexed.
++
+.Properties of objects in `in_progress`
+[%collapsible%open]
+=====
+`index`::
+(string) The name of the source backing index.
+
+`total_doc_count`::
+(integer) The number of documents in the source backing index.
+
+`reindexed_doc_count`::
+(integer) The number of documents which have already been added to the destination backing index.
+=====
+
+`pending`::
+(integer) The number of backing indices which still need to be upgraded and have not yet been started.
+
+`errors`::
+(array of objects) Information on any errors which have occurred.
++
+.Properties of objects in `errors`
+[%collapsible%open]
+=====
+`index`::
+(string) The name of a backing index which has had an error during reindex.
+
+`message`::
+(string) Description of the error.
+=====
+
+`exceptions`::
+(Optional, string)
+Exception message for a reindex failure if the failure could not be tied to a particular index.
+
+
+[[data-stream-reindex-status-example]]
+==== {api-examples-title}
+
+[source,console]
+----
+GET _migration/reindex/my-data-stream/_status
+----
+
+The following is a typical response:
+[source,console-result]
+----
+{
+  "start_time_millis": 1737676174349,
+  "complete": false,
+  "total_indices_in_data_stream": 4,
+  "total_indices_requiring_upgrade": 3,
+  "successes": 1,
+  "in_progress": [
+    {
+      "index": ".ds-my-data-stream-2025.01.23-000002",
+      "total_doc_count": 10000000,
+      "reindexed_doc_count": 1000
+    }
+  ],
+  "pending": 1,
+  "errors": []
+}
+----
+// TEST[skip:cannot easily clean up reindex task between tests]
+
+For a more in-depth example showing the usage of this API along with the <<data-stream-reindex-api,reindex>> and <<data-stream-reindex-cancel-api,cancel>> APIs,
+see this <<reindex-data-stream-api-example,example>>.