Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating coordinator stats branch with main. #6

Merged
merged 98 commits into from
Sep 11, 2023

Conversation

sgup432
Copy link
Collaborator

@sgup432 sgup432 commented Sep 11, 2023

Description

[Describe what this change achieves]

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

linuxpi and others added 30 commits August 28, 2023 08:31
…thing specified by user (opensearch-project#9264)

---------

Signed-off-by: bansvaru <[email protected]>
Signed-off-by: Varun Bansal <[email protected]>
)

* Add support to use trace propagated from client

Signed-off-by: Gagan Juneja <[email protected]>

* Add support to use trace propagated from client

Signed-off-by: Gagan Juneja <[email protected]>

* Refactor code

Signed-off-by: Gagan Juneja <[email protected]>

* Add support to use trace propagated from client

Signed-off-by: Gagan Juneja <[email protected]>

* Add support to use trace propagated from client

Signed-off-by: Gagan Juneja <[email protected]>

* Refactor code

Signed-off-by: Gagan Juneja <[email protected]>

* Refactor code

Signed-off-by: Gagan Juneja <[email protected]>

* Merged CHANGELOG

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

---------

Signed-off-by: Gagan Juneja <[email protected]>
Signed-off-by: Gagan Juneja <[email protected]>
Co-authored-by: Gagan Juneja <[email protected]>
* Update BWC testing documentation

Signed-off-by: Stephen Crawford <[email protected]>

* Fix code block

Signed-off-by: Stephen Crawford <[email protected]>

* Fix code block

Signed-off-by: Stephen Crawford <[email protected]>

* Add placeholder

Signed-off-by: Stephen Crawford <[email protected]>

* Fix typos

Signed-off-by: Stephen Crawford <[email protected]>

* Fix table of contents order

Signed-off-by: Stephen Crawford <[email protected]>

* Add colon

Signed-off-by: Stephen Crawford <[email protected]>

* trigger retry

Signed-off-by: Stephen Crawford <[email protected]>

---------

Signed-off-by: Stephen Crawford <[email protected]>
…pensearch-project#9562)

* For sort request on timeseries field use non concurrent search path

Signed-off-by: Sorabh Hamirwasia <[email protected]>

* Address review feedback

Signed-off-by: Sorabh Hamirwasia <[email protected]>

---------

Signed-off-by: Sorabh Hamirwasia <[email protected]>
…ing engine reset. (opensearch-project#9495)

* Fix bug where replica shows stale doc count during engine reset.

This change fixes an issue where replica shards can temporarily return stale results while converting to a RO engine during an engine reset.  This is possible because NRTReplicationEngine did not previously implement flush and the freshest data is only active on the reader. Fixed by implementing flush and also honoring acquireLatestCommit's flushFirst parameter.

Signed-off-by: Marc Handalian <[email protected]>

* Add changelog entry.

Signed-off-by: Marc Handalian <[email protected]>

* Add unit test for search during engine reset.

Signed-off-by: Marc Handalian <[email protected]>

* Remove useless test.

Signed-off-by: Marc Handalian <[email protected]>

---------

Signed-off-by: Marc Handalian <[email protected]>
…plication (opensearch-project#9593)

* [Segment Replication] Add ClusterState utility to identify SEGMENT replication

Signed-off-by: Suraj Singh <[email protected]>

* Address review comment

Signed-off-by: Suraj Singh <[email protected]>

* Address review comments

Signed-off-by: Suraj Singh <[email protected]>

---------

Signed-off-by: Suraj Singh <[email protected]>
opensearch-project#9578)

* Bump io.grpc:grpc-api from 1.57.1 to 1.57.2 in /plugins/repository-gcs

Bumps [io.grpc:grpc-api](https://github.com/grpc/grpc-java) from 1.57.1 to 1.57.2.
- [Release notes](https://github.com/grpc/grpc-java/releases)
- [Commits](grpc/grpc-java@v1.57.1...v1.57.2)

---
updated-dependencies:
- dependency-name: io.grpc:grpc-api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* Add SearchExtBuilders to SearchResponse. [Issue opensearch-project#9328](opensearch-project#9328)

Signed-off-by: Austin Lee <[email protected]>

* Keep SearchResponse immutable, add a constructor to take a List of SearchExtBuilders.

Signed-off-by: Austin Lee <[email protected]>

* Fix spotlessJavaCheck findings.

Signed-off-by: Austin Lee <[email protected]>

* Move SearchExtBuilders into SearchResponseSections, fix indenting in SearchRequest.

Signed-off-by: Austin Lee <[email protected]>

* Updated changelog (mixed minor formatting issues), added version checks on serialization/deserialization, added a Builder for making copies of SearchResponse easier.

Signed-off-by: Austin Lee <[email protected]>

* Add GenericSearchExtBuilder as a catch-all for SearchExtBuilders not registered in xcontent registry.

Signed-off-by: Austin Lee <[email protected]>

* Simplify GenericSearchExtBuilder using a single Object member.

Signed-off-by: Austin Lee <[email protected]>

* Address additional review comments.

Signed-off-by: Austin Lee <[email protected]>

* Add Javadocs.

Signed-off-by: Austin Lee <[email protected]>

---------

Signed-off-by: Austin Lee <[email protected]>
…tations (part 1) (opensearch-project#9520)

* [BWC and API enforcement] Decorate the existing APIs with proper annotations (part 1)

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>
…tion. (opensearch-project#9471)

* Fix SegmentReplicationUsingRemoteStoreIT#testDropPrimaryDuringReplication.

This test is failing because a concurrent flush can wipe out an old commit file
while we are in the remote store refresh listener. The listener will fetch the latest infos from the reader which will reference a segments_n tht has been deleted by an incoming flush.

To fix this, InternalEngine will preserve the latest commit until a new commit is loaded onto the readerManager.

Signed-off-by: Marc Handalian <[email protected]>

* update InternalEngine to preserve commit file until a new commit is refreshed on.

Signed-off-by: Marc Handalian <[email protected]>

* Update ReadOnlyEngine inside of resetEngineToGlobalCheckpoint to implement getSegmentInfosSnapshot.
This ensures access to this function is not permitted on the ReadOnlyEngine and is delegated to the new IE once opened.

Signed-off-by: Marc Handalian <[email protected]>

* Update javadoc.

Signed-off-by: Marc Handalian <[email protected]>

* spotless.

Signed-off-by: Marc Handalian <[email protected]>

---------

Signed-off-by: Marc Handalian <[email protected]>
…n before validating spans (opensearch-project#9561)

* Allow MockTracingTelemetry to await for asynchronous tasks termination before validating spans

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>
…pensearch-project#9426)

Benchmarks show that ArrayList performs better than TreeSet. Added a comment
on where to find the results and benchmark.

Signed-off-by: Kiran Reddy <[email protected]>
…ch-project#9536)

* pick oldest OS version replica to promote as primary

Signed-off-by: Poojita Raj <[email protected]>

* add test

Signed-off-by: Poojita Raj <[email protected]>

* refactor

Signed-off-by: Poojita Raj <[email protected]>

* refactor to avoid coupling

Signed-off-by: Poojita Raj <[email protected]>

* add comments

Signed-off-by: Poojita Raj <[email protected]>

---------

Signed-off-by: Poojita Raj <[email protected]>
…h-project#9507)

* Decouple replication lag from replication timer logic used to fail stale replicas

Signed-off-by: Ankit Kala <[email protected]>

* Added changelog entry

Signed-off-by: Ankit Kala <[email protected]>

* Addressed comments

Signed-off-by: Ankit Kala <[email protected]>

* Addressed comments 2

Signed-off-by: Ankit Kala <[email protected]>

* Addressed comments

Signed-off-by: Ankit Kala <[email protected]>

* Retry gradle

Signed-off-by: Ankit Kala <[email protected]>

* fix UT

Signed-off-by: Ankit Kala <[email protected]>

* Addressed comments

Signed-off-by: Ankit Kala <[email protected]>

* Retry Gradle

Signed-off-by: Ankit Kala <[email protected]>

---------

Signed-off-by: Ankit Kala <[email protected]>
opensearch-project#9160)

* Upload all index metadata to remote store using BlobStoreRepository interface

Signed-off-by: Sooraj Sinha <[email protected]>
…Key Provider Plugin to provide keys (opensearch-project#8466)


Signed-off-by: Vikas Bansal <[email protected]>
Signed-off-by: Gaurav Bafna <[email protected]>
* Expose DelimitedTermFrequencyTokenFilter

Relates: opensearch-project#9413

This commit exposes Lucene's delimited term frequency token filter to be
able to provide term frequencies along with terms.

Signed-off-by: Russ Cam <[email protected]>

* fix format violations

Signed-off-by: Russ Cam <[email protected]>

* fix test and add to changelog

Signed-off-by: Russ Cam <[email protected]>

* Address PR feedback

- Add unit tests for DelimitedTermFrequencyTokenFilterFactory
- Remove IllegalArgumentException as caught exception
- Add skip to yaml rest tests to skip for version < 2.10

Signed-off-by: Russ Cam <[email protected]>

* formatting

Signed-off-by: Russ Cam <[email protected]>

* Rename filter

Signed-off-by: Russ Cam <[email protected]>

* update naming in REST tests

Signed-off-by: Russ Cam <[email protected]>

---------

Signed-off-by: Russ Cam <[email protected]>
…t referenced by reader with matching checksum (opensearch-project#9630)

* [Segment Replication] Allow segment replication with on disk files not referenced by reader with matching checksum

Signed-off-by: Suraj Singh <[email protected]>

* Address review comments

Signed-off-by: Suraj Singh <[email protected]>

* Address review comments

Signed-off-by: Suraj Singh <[email protected]>

* Spotless fix

Signed-off-by: Suraj Singh <[email protected]>

---------

Signed-off-by: Suraj Singh <[email protected]>
…ation (opensearch-project#9617)

* Add SpanBuilder support

Signed-off-by: Gagan Juneja <[email protected]>

* Refactor code

Signed-off-by: Gagan Juneja <[email protected]>

* Redefine telemetry context restoration

Signed-off-by: Gagan Juneja <[email protected]>

* Update changelog

Signed-off-by: Gagan Juneja <[email protected]>

* Stores the SpanScope in ThreadLocal

Signed-off-by: Gagan Juneja <[email protected]>

* Revert the context name changes

Signed-off-by: Gagan Juneja <[email protected]>

* Change the span::endSpan and SpanScope::close behaviour

Signed-off-by: Gagan Juneja <[email protected]>

* Supressed warnings

Signed-off-by: Gagan Juneja <[email protected]>

* Add more test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Fix java doc

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Fix failing test

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

---------

Signed-off-by: Gagan Juneja <[email protected]>
Signed-off-by: Gagan Juneja <[email protected]>
Co-authored-by: Gagan Juneja <[email protected]>
sachinpkale and others added 29 commits September 7, 2023 16:35
…opensearch-project#9796)

* Decrease store refcount on any failure to create NRTReplicationEngine

Signed-off-by: Sachin Kale <[email protected]>

* Add unit tests

Signed-off-by: Sachin Kale <[email protected]>

* Fix spotless

Signed-off-by: Sachin Kale <[email protected]>

---------

Signed-off-by: Sachin Kale <[email protected]>
Co-authored-by: Sachin Kale <[email protected]>
* Fix default tracer falky test

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

---------

Signed-off-by: Gagan Juneja <[email protected]>
Co-authored-by: Gagan Juneja <[email protected]>
…pensearch-project#9727)

* Improve performance of rounding dates in date_histogram aggregation

Signed-off-by: Ketan Verma <[email protected]>

* Minor refactoring changes

Signed-off-by: Ketan Verma <[email protected]>

---------

Signed-off-by: Ketan Verma <[email protected]>
…peTestCase annotation (opensearch-project#9916)

Signed-off-by: Neetika Singhal <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
…nt Replication (opensearch-project#9585)

* support realtime TermVector and MultiTermVector requests with segment replication.

Signed-off-by: Rishikesh1159 <[email protected]>

* Fix TermVector requests with segrep.

Signed-off-by: Rishikesh1159 <[email protected]>

* Refacotring.

Signed-off-by: Rishikesh1159 <[email protected]>

* Address comments on PR.

Signed-off-by: Rishikesh1159 <[email protected]>

---------

Signed-off-by: Rishikesh1159 <[email protected]>
Signed-off-by: Rishikesh Pasham <[email protected]>
…ject#9415)

* Add Instrumentation

Signed-off-by: Gagan Juneja <[email protected]>

* update changelog

Signed-off-by: Gagan Juneja <[email protected]>

* clear the context

Signed-off-by: Gagan Juneja <[email protected]>

* Add Instrumentation

Signed-off-by: Gagan Juneja <[email protected]>

* Fix test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Add Instrumentation

Signed-off-by: Gagan Juneja <[email protected]>

* Add Instrumentation

Signed-off-by: Gagan Juneja <[email protected]>

* Remove internal transport action check

Signed-off-by: Gagan Juneja <[email protected]>

* Changes

Signed-off-by: Gagan Juneja <[email protected]>

* Refactor code

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Fix failing test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Fix failing test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Fix failing test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Fix failing test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Fix failing test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Fix failing test cases

Signed-off-by: Gagan Juneja <[email protected]>

* Fix java doc

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Address merge conflicts

Signed-off-by: Gagan Juneja <[email protected]>

* Address merge conflicts

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Enable NumberOfTraceIDsEqualToRequests test

Signed-off-by: Gagan Juneja <[email protected]>

* Fix java doc

Signed-off-by: Gagan Juneja <[email protected]>

* Fix java doc

Signed-off-by: Gagan Juneja <[email protected]>

* Empty-Commit

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Address review comment

Signed-off-by: Gagan Juneja <[email protected]>

* Fix test case

Signed-off-by: Gagan Juneja <[email protected]>

* Remove instrumentation from TransportService

Signed-off-by: Gagan Juneja <[email protected]>

---------

Signed-off-by: Gagan Juneja <[email protected]>
Signed-off-by: Gagan Juneja <[email protected]>
Co-authored-by: Gagan Juneja <[email protected]>
This plugin has been moved to its own [repository][1].

[1]: https://github.com/opensearch-project/custom-codecs

Signed-off-by: Andrew Ross <[email protected]>
* Clear CHANGELOG for released 2.x changes

Signed-off-by: Kunal Kotwani <[email protected]>

* Add release notes for 2.10

Signed-off-by: Kunal Kotwani <[email protected]>

---------

Signed-off-by: Kunal Kotwani <[email protected]>
…ject#9956)

Fix is pending in opensearch-project#7771, but that PR may take some time to land in main
so muting for the time being.

Signed-off-by: Andrew Ross <[email protected]>
Instead of appending a new comment every time check compatibility runs,
this will instead update a single comment with the latest results. The
history of previous runs is retained in the comment as GitHub allows you
to view the edited revisions of any comment. This matches the behavior
of the codecov workflow and should substantially reduce noise in PRs.

Relates opensearch-project#9699

Signed-off-by: Andrew Ross <[email protected]>
* fixing javadoc of org.opensearch.cli.Terminal

Signed-off-by: Laurent Laborde <[email protected]>

* fixing javadoc of org.opensearch.cli.ExitCodes

Signed-off-by: Laurent Laborde <[email protected]>

* fixing javadoc

Signed-off-by: Laurent Laborde <[email protected]>

* fixing a mishap in formatting rule

Signed-off-by: Laurent Laborde <[email protected]>

---------

Signed-off-by: Laurent Laborde <[email protected]>
…ject#9575)

* Bump peter-evans/create-or-update-comment from 2 to 3

Bumps [peter-evans/create-or-update-comment](https://github.com/peter-evans/create-or-update-comment) from 2 to 3.
- [Release notes](https://github.com/peter-evans/create-or-update-comment/releases)
- [Commits](peter-evans/create-or-update-comment@v2...v3)

---
updated-dependencies:
- dependency-name: peter-evans/create-or-update-comment
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* Bump actions/checkout from 2 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
@sgup432 sgup432 merged commit 0e44b49 into buddharajusahil:CoordinatorStats Sep 11, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.