Add batch async shard fetch transport action for replica shards

[sudarshan-baliga]

Description

This pull request is part of the improvement #5098
This PR for adding the transport action for the async fetching of the shard information for batch of replica shards from the nodes in a single request per node. Code is inspired by the existing transport action TransportNodesListShardStoreMetadata.java which fetches just one shard information from the node per request.

• Added Transport action TransportNodesBatchListShardStoreMetadata.java for fetching information about batch of replica shards.
• Added helper class TransportNodesListShardStoreMetadataHelper.java for reducing code duplication.
• Similar pull request for primary shard allocator transport action Added transport action for bulk async shard fetch for primary shards #8218
  Tested the changes manually by changing the replica shard allocator in the gateway allocator in my custom branch sudarshan-baliga@0919259 and verified that the batch of shard information was fetched while adding and removing the node from the cluster.

Data node log which was dropped and readded

 ] [data2] started
[2023-06-29T20:29:28,613][INFO ][o.o.i.s.TransportNodesBatchListShardStoreMetadata] [data2] Loaded store meta data for {[my-index-4][0]=[[StoreFilesMetadata{, shardId=[my-index-4][0], metadataSnapshot{size=1, syncId=null}}]], [my-index-4][1]=[[StoreFilesMetadata{, shardId=[my-index-4][1], metadataSnapshot{size=1, syncId=null}}]]} shards

Master node log when the data node was readded

[2023-06-29T20:29:29,074][INFO ][o.o.g.G.TestInternalReplicaShardAllocator] [master1] sdarbStore replica data {{data1}{PSqG-WkGSD2WiCi3dvXccw}{J84l7dayS2mx-8ocMXMn_Q}{127.0.0.1}{127.0.0.1:9301}{dir}{shard_indexing_pressure_enabled=true}=[[{data1}{PSqG-WkGSD2WiCi3dvXccw}{J84l7dayS2mx-8ocMXMn_Q}{127.0.0.1}{127.0.0.1:9301}{dir}{shard_indexing_pressure_enabled=true}][StoreFilesMetadata{, shardId=[my-index-4][0], metadataSnapshot{size=1, syncId=null}}]], {data2}{EMGkWoq8Q2mK3zVXFyYK0g}{fW3xmnx7S16uJCiI9RjM9A}{127.0.0.1}{127.0.0.1:9302}{dir}{shard_indexing_pressure_enabled=true}=[[{data2}{EMGkWoq8Q2mK3zVXFyYK0g}{fW3xmnx7S16uJCiI9RjM9A}{127.0.0.1}{127.0.0.1:9302}{dir}{shard_indexing_pressure_enabled=true}][StoreFilesMetadata{, shardId=[my-index-4][0], metadataSnapshot{size=1, syncId=null}}]]}
[2023-06-29T20:29:29,438][INFO ][o.o.c.r.a.AllocationService] [master1] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[my-index-4][1], [my-index-4][0]]]).
[2023-06-29T20:29:29,734][INFO ][o.o.g.GatewayAllocator   ] [master1] sdarbStore Collecting of total shards ={}, over transport

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/18713/
• CommitID: 27c4e2a
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?