Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PrimaryShardAllocator refactor to abstract out shard state and method calls #9760

Merged
merged 23 commits into from
Mar 12, 2024

Conversation

shiv0408
Copy link
Member

@shiv0408 shiv0408 commented Sep 5, 2023

Description

In this PR, we are refactoring the PrimaryShardAllocator code to make it more reusable as we bring in Batch Allocator in #8916.

We have broken this function to multiple functions, so that part of codes can be reused and only a single function need to be overwritten in child class to implement the Batch allocator.

public AllocateUnassignedDecision makeAllocationDecision(
final ShardRouting unassignedShard,
final RoutingAllocation allocation,
final Logger logger
) {

Created new internal classes NodeShardState and NodeShardStates to encapsulate the data that needs to be passed around in these functions.

Related Issues

Resolves #8960

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2023

Compatibility status:

Checks if related components are compatible with change 61ec18c

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/sql.git]

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT:
  • URL:
  • CommitID: 41fcdca
    Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
    Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT:
  • URL:
  • CommitID: 4dd5219
    Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
    Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2023

Compatibility status:

Checks if related components are compatible with change 4dd5219

Incompatible components

Skipped components

Compatible components

@amkhar
Copy link
Contributor

amkhar commented Sep 6, 2023

Thanks @shiv0408 for making this code reusable.
One request - please keep only PSA refactoring related change in this PR. Remove the draft tag then.
I'll update my own draft PR according to this PR 👍🏼

@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2023

Compatibility status:

Checks if related components are compatible with change 8a25e7f

Incompatible components

Skipped components

Compatible components

@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2023

Gradle Check (Jenkins) Run Completed with:

Copy link
Contributor

❌ Gradle check result for 2aa0845: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for c4454dd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 1ec2ba1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@opensearch-trigger-bot
Copy link
Contributor

This PR is stalled because it has been open for 30 days with no activity.

@opensearch-trigger-bot opensearch-trigger-bot bot added the stalled Issues that have stalled label Feb 4, 2024
Copy link
Contributor

github-actions bot commented Feb 5, 2024

❌ Gradle check result for fd9b215: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Copy link
Contributor

github-actions bot commented Feb 5, 2024

❌ Gradle check result for d772499: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Feb 5, 2024

❌ Gradle check result for a6fb7fc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Feb 5, 2024

✅ Gradle check result for 61ec18c: SUCCESS

@opensearch-trigger-bot opensearch-trigger-bot bot removed the stalled Issues that have stalled label Feb 5, 2024
@shwetathareja shwetathareja merged commit afd3969 into opensearch-project:main Mar 12, 2024
29 checks passed
@shiv0408 shiv0408 deleted the batched-psa branch March 12, 2024 20:49
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Mar 18, 2024
… calls (opensearch-project#9760)

* PrimaryShardAllocator refactor to abstract out shard state and method calls

Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
@amkhar amkhar added the backport 2.x Backport to 2.x branch label Mar 19, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Mar 19, 2024
… calls (#9760)

* PrimaryShardAllocator refactor to abstract out shard state and method calls

Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
(cherry picked from commit afd3969)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
shwetathareja pushed a commit that referenced this pull request Mar 19, 2024
… calls (#9760) (#12762)

* PrimaryShardAllocator refactor to abstract out shard state and method calls



(cherry picked from commit afd3969)

Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
shiv0408 added a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
… calls (opensearch-project#9760)

* PrimaryShardAllocator refactor to abstract out shard state and method calls

Signed-off-by: Shivansh Arora <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
@shiv0408 shiv0408 self-assigned this Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch bug Something isn't working Cluster Manager skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] Add a batch allocator for building allocation decisions for multiple primary & replica shards
5 participants