[Backport 2.x] Add support download latest index metadata from remote

[opensearch-trigger-bot]

Backport 422dd3c from #9477.

[github-actions]

Compatibility status:

Checks if related components are compatible with change 33fa774

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/24236/
• CommitID: 33fa774
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[linuxpi]

Test Failures

org.opensearch.indices.replication.SegmentReplicationSuiteIT.testFullRestartDuringReplication - Flaky - #9499
org.opensearch.indices.replication.SegmentReplicationSuiteIT.classMethod
org.opensearch.snapshots.CloneSnapshotIT.testCloneShallowSnapshotIndex -- Flaky #8279
org.opensearch.snapshots.CloneSnapshotIT.testCloneShallowSnapshotIndex
org.opensearch.snapshots.CloneSnapshotIT.testCloneShallowSnapshotIndex

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.testReplicaPromotion

• URL: https://build.ci.opensearch.org/job/gradle-check/24305/
• CommitID: 33fa774
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Merging #9713 (33fa774) into 2.x (58ae69d) will increase coverage by 0.15%.
Report is 3 commits behind head on 2.x.
The diff coverage is 82.05%.

@@             Coverage Diff              @@
##                2.x    #9713      +/-   ##
============================================
+ Coverage     70.80%   70.95%   +0.15%     
- Complexity    57868    57919      +51     
============================================
  Files          4793     4793              
  Lines        273397   273428      +31     
  Branches      40216    40219       +3     
============================================
+ Hits         193583   194022     +439     
+ Misses        63235    62807     -428     
- Partials      16579    16599      +20     

Files Changed	Coverage Δ
...ch/repositories/blobstore/BlobStoreRepository.java	59.38% <0.00%> (+0.16%)	⬆️
...rg/opensearch/repositories/s3/S3BlobContainer.java	80.00% <33.33%> (-0.14%)	⬇️
...org/opensearch/common/blobstore/BlobContainer.java	73.68% <66.66%> (-7.57%)	⬇️
...arch/gateway/remote/RemoteClusterStateService.java	80.92% <93.10%> (+2.45%)	⬆️

... and 455 files with indirect coverage changes