From c5c2fa63c86faca0144b4ef00427df23d6bdf3ae Mon Sep 17 00:00:00 2001 From: David Venable Date: Wed, 30 Oct 2024 12:49:18 -0500 Subject: [PATCH] Renames the project to OpenSearch Data Prepper. (#5128) This PR updates a few key files to call the project OpenSearch Data Prepper over Data Prepper. Signed-off-by: David Venable --- README.md | 14 ++++++------ RELEASING.md | 22 +++++++++---------- TRIAGING.md | 20 ++++++++--------- docs/developer_guide.md | 48 ++++++++++++++++++++--------------------- 4 files changed, 52 insertions(+), 52 deletions(-) diff --git a/README.md b/README.md index b53cefadc5..fefcac810e 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,24 @@ -Data Prepper +OpenSearch Data Prepper [![codecov](https://codecov.io/gh/opensearch-project/data-prepper/branch/main/graph/badge.svg?token=IS7GOIY622)](https://codecov.io/gh/opensearch-project/data-prepper) -# Data Prepper +# OpenSearch Data Prepper -We envision Data Prepper as an open source data collector for observability data (trace, logs, metrics) that can filter, enrich, transform, normalize, and aggregate data for downstream analysis and visualization. It will support stateful processing across multiple instances of data pipelines for observability use cases such as distributed tracing and multi-line log events (e.g. stack traces, aggregations, and log-to-metric transformations). Currently Data Prepper supports processing of distributed trace data and log ingestion with plans to support metric data in the future. +We envision OpenSearch Data Prepper as an open source data collector for observability data (trace, logs, metrics) that can filter, enrich, transform, normalize, and aggregate data for downstream analysis and visualization. It will support stateful processing across multiple instances of data pipelines for observability use cases such as distributed tracing and multi-line log events (e.g. stack traces, aggregations, and log-to-metric transformations). Currently OpenSearch Data Prepper supports processing of distributed trace data and log ingestion with plans to support metric data in the future. -Please read the [Overview](docs/overview.md) to understand what Data Prepper is and how it works. +Please read the [Overview](docs/overview.md) to understand what OpenSearch Data Prepper is and how it works. ## Getting Started -Our [Getting Started](docs/getting_started.md) guide is the best starting point for anybody who wants to run Data Prepper. +Our [Getting Started](docs/getting_started.md) guide is the best starting point for anybody who wants to run OpenSearch Data Prepper. -Please read the [Trace Analytics](docs/trace_analytics.md) guide or [Log Analytics](docs/log_analytics.md) to get started with using Data Prepper for trace or log analytics use cases. +Please read the [Trace Analytics](docs/trace_analytics.md) guide or [Log Analytics](docs/log_analytics.md) to get started with using OpenSearch Data Prepper for trace or log analytics use cases. ## Project Resources * [Downloads](https://opensearch.org/downloads.html) * [Documentation](https://opensearch.org/docs/latest/clients/data-prepper/index/) * [Configuration Reference](https://opensearch.org/docs/latest/clients/data-prepper/data-prepper-reference/) -* Need help? Try the [Data Prepper category](https://discuss.opendistrocommunity.dev/c/data-prepper/61) in the OpenSearch forums +* Need help? Try the [OpenSearch Data Prepper category](https://discuss.opendistrocommunity.dev/c/data-prepper/61) in the OpenSearch forums * [Project Principles](https://opensearch.org/#principles) ## Contribute diff --git a/RELEASING.md b/RELEASING.md index 2d51eb42db..2fa90f57b4 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,7 +1,7 @@ # Releasing -This document outlines the process for releasing Data Prepper. -It is a guide for maintainers of the Data Prepper project to release a new version. +This document outlines the process for releasing OpenSearch Data Prepper. +It is a guide for maintainers of the OpenSearch Data Prepper project to release a new version. ## Overview @@ -16,13 +16,13 @@ This document has three broad categories of steps to follow: ### Branch setup -Data Prepper uses a release branch for releasing. +OpenSearch Data Prepper uses a release branch for releasing. The [Developer Guide](docs/developer_guide.md#backporting) discusses this in detail. The repository has a release branch for a major/minor version. Patch versions will continue on the same branch. -For example, Data Prepper `2.6.0` was released from the `2.6` branch. -Additionally, Data Prepper `2.6.1` and `2.6.2` were also released from the `2.6` branch. +For example, OpenSearch Data Prepper `2.6.0` was released from the `2.6` branch. +Additionally, OpenSearch Data Prepper `2.6.1` and `2.6.2` were also released from the `2.6` branch. If you are creating a new major/minor release, then you will need to create the release branch. Use GitHub to create a new branch. @@ -42,7 +42,7 @@ Steps: ### Update versions -The Data Prepper version is defined in the [`gradle.properties`](https://github.com/opensearch-project/data-prepper/blob/main/gradle.properties) file. +The OpenSearch Data Prepper version is defined in the [`gradle.properties`](https://github.com/opensearch-project/data-prepper/blob/main/gradle.properties) file. We must update this whenever we create a new release. We will need two PRs to update it. @@ -76,7 +76,7 @@ Note: This step can be automated through [#4877](https://github.com/opensearch-p ### Update the THIRD-PARTY file We should update the `THIRD-PARTY` file for every release. -Data Prepper has a GitHub action that will generate this and create a PR with the updated file. +OpenSearch Data Prepper has a GitHub action that will generate this and create a PR with the updated file. Steps: * Go the [Third Party Generate action](https://github.com/opensearch-project/data-prepper/actions/workflows/third-party-generate.yml) @@ -118,10 +118,10 @@ Also tag this with the `backport {major}.{minor}` to create a PR that you can me ## Performing a release -This section outlines how to perform a Data Prepper release using GitHub Actions and the OpenSearch build infrastructure. -The audience for this section are Data Prepper maintainers. +This section outlines how to perform a OpenSearch Data Prepper release using GitHub Actions and the OpenSearch build infrastructure. +The audience for this section are OpenSearch Data Prepper maintainers. -### Start the release Data Prepper action +### Start the release OpenSearch Data Prepper action To run the release, go to the [Release Artifacts](https://github.com/opensearch-project/data-prepper/actions/workflows/release.yml) GitHub Action. @@ -159,7 +159,7 @@ This includes building the artifacts, testing, drafting a GitHub release, and pr The release build will create a new GitHub issue requesting to release the project. This needs two maintainers to approve. -To approve, load [Data Prepper issues](https://github.com/opensearch-project/data-prepper/issues). +To approve, load [OpenSearch Data Prepper issues](https://github.com/opensearch-project/data-prepper/issues). Look for and open a new issue starting with _Manual approval required for workflow_. Verify that the metadata looks correct and that we want to release. Add a new comment on the issue with the word _approve_ or _approved_ in it. diff --git a/TRIAGING.md b/TRIAGING.md index ba20857061..216c71083b 100644 --- a/TRIAGING.md +++ b/TRIAGING.md @@ -1,8 +1,8 @@ -Data Prepper +OpenSearch Data Prepper -The Data Prepper maintainers seek to promote an inclusive and engaged community of contributors. +The OpenSearch Data Prepper maintainers seek to promote an inclusive and engaged community of contributors. In order to facilitate this, weekly triage meetings are open to all and attendance is encouraged for anyone who hopes to contribute, discuss an issue, or learn more about the project. -To learn more about contributing to the Data Prepper project visit the [Contributing](./CONTRIBUTING.md) documentation. +To learn more about contributing to the OpenSearch Data Prepper project visit the [Contributing](./CONTRIBUTING.md) documentation. ### Do I need to attend for my issue to be addressed/triaged? @@ -17,7 +17,7 @@ However, should we run out of time before your issue is discussed, you are alway ### How do I join the triage meeting? Meetings are hosted regularly Tuesdays at 2:30 PM US Central Time (12:30 PM Pacific Time) and can be joined via the links posted on the [OpenSearch Meetup Group](https://www.meetup.com/opensearch/events/) list of events. -The event will be titled `Data Prepper Triage Meeting`. +The event will be titled `OpenSearch Data Prepper Triage Meeting`. After joining the video meeting, you can enable your video / voice to join the discussion. If you do not have a webcam or microphone available, you can still join in via the text chat. @@ -28,9 +28,9 @@ If you have an issue you'd like to bring forth please consider getting a link to Meetings are 30 minutes and structured as follows: -1. Initial Gathering: As we gather, feel free to turn on video and engage in informal and open-to-all conversation. A volunteer Data Prepper maintainer will share the [Data Prepper Tracking Board](https://github.com/orgs/opensearch-project/projects/82/) and proceed. +1. Initial Gathering: As we gather, feel free to turn on video and engage in informal and open-to-all conversation. A volunteer OpenSearch Data Prepper maintainer will share the [Data Prepper Tracking Board](https://github.com/orgs/opensearch-project/projects/82/) and proceed. 2. Announcements: We will make any announcements at the beginning, if necessary. -3. Untriaged issues: We will review all untriaged [issues](https://github.com/orgs/opensearch-project/projects/82/views/6) for the Data Prepper repository. If you have an item here, you may spend a few minutes to explain your request. +3. Untriaged issues: We will review all untriaged [issues](https://github.com/orgs/opensearch-project/projects/82/views/6) for the OpenSearch Data Prepper repository. If you have an item here, you may spend a few minutes to explain your request. 4. Member Requests: Opportunity for any meeting member to ask for consideration of an issue or pull request. 5. Release review: If time permits, and we find it necessary, we will review [items for the current release](https://github.com/orgs/opensearch-project/projects/82/views/14). 6. Follow-up issues: If time permits, we will review the [follow up items](https://github.com/orgs/opensearch-project/projects/82/views/18). @@ -46,15 +46,15 @@ Attending the triage meetings is a great way for a new contributor to learn abou If you have an existing issue you would like to discuss, you can always comment on the issue itself. Alternatively, you are welcome to come to the triage meeting to discuss. -### Is this meeting a good place to get help using Data Prepper? +### Is this meeting a good place to get help using OpenSearch Data Prepper? -While we are always happy to help the community, the best resource for usage questions is the [the Data Prepper discussion forum](https://github.com/opensearch-project/data-prepper/discussions) on GitHub. +While we are always happy to help the community, the best resource for usage questions is the [OpenSearch Data Prepper discussion forum](https://github.com/opensearch-project/data-prepper/discussions) on GitHub. -There you can find answers to many common questions as well as speak with implementation experts and Data Prepper maintainers. +There you can find answers to many common questions as well as speak with implementation experts and OpenSearch Data Prepper maintainers. ### What are the issue labels associated with triaging? -There are several labels that are particularly important for triaging in Data Prepper: +There are several labels that are particularly important for triaging in OpenSearch Data Prepper: | Label | When applied | Meaning | | ----- | ------------ | ------- | diff --git a/docs/developer_guide.md b/docs/developer_guide.md index 098ee5a0d3..7ced290dfd 100644 --- a/docs/developer_guide.md +++ b/docs/developer_guide.md @@ -1,16 +1,16 @@ -# Data Prepper Developer Guide +# OpenSearch Data Prepper Developer Guide -This page is for anybody who wishes to contribute code to Data Prepper. Welcome! +This page is for anybody who wishes to contribute code to OpenSearch Data Prepper. Welcome! ## Contributions -First, please read our [contribution guide](../CONTRIBUTING.md) for more information on how to contribute to Data Prepper. +First, please read our [contribution guide](../CONTRIBUTING.md) for more information on how to contribute to OpenSearch Data Prepper. ## Installation Prerequisites ### Java Versions -Building Data Prepper requires JDK 11 or 17. The Data Prepper Gradle build runs in a Java 11 or 17 JVM, but uses +Building OpenSearch Data Prepper requires JDK 11 or 17. The OpenSearch Data Prepper Gradle build runs in a Java 11 or 17 JVM, but uses [Gradle toolchains](https://docs.gradle.org/current/userguide/toolchains.html) to compile the Java code using Java 11. If you have a JDK 11 installed locally, Gradle will use your installed JDK 11. If you do not, Gradle will install JDK 11. @@ -22,7 +22,7 @@ All main source code builds on JDK 11, so it must be compatible with Java 11. Th ## Building from source The assemble task will build the Jar files and create a runnable distribution without running the integration -tests. If you are just looking to build Data Prepper from source, this build is faster than running the integration test suite. +tests. If you are just looking to build OpenSearch Data Prepper from source, this build is faster than running the integration test suite. To build the project from source, run the following command from the project root: @@ -46,9 +46,9 @@ To build, run the following command from the project root: ## Running the project -Before running Data Prepper, check that configuration files (see [configuration](configuration.md) docs for more -information) have been put in the respective folders under Data Prepper home directory. When building from source, -Data Prepper home directory is at `release/archives/linux/build/install/opensearch-data-prepper-$VERSION-linux-x64` +Before running OpenSearch Data Prepper, check that configuration files (see [configuration](configuration.md) docs for more +information) have been put in the respective folders under OpenSearch Data Prepper home directory. When building from source, +OpenSearch Data Prepper home directory is at `release/archives/linux/build/install/opensearch-data-prepper-$VERSION-linux-x64` ($VERSION is the current version as defined in [gradle.properties](../gradle.properties)). The configuration files should be put in the following folders: @@ -60,7 +60,7 @@ Go to home directory: cd release/archives/linux/build/install/opensearch-data-prepper-$VERSION-linux-x64 ``` -Data Prepper can then be run with the following commands: +OpenSearch Data Prepper can then be run with the following commands: ``` bin/data-prepper ``` @@ -77,7 +77,7 @@ Additionally, Log4j 2 configuration file is read from `config/log4j2.properties` ## Building & Running the Docker Image In some cases, you may wish to build a local Docker image and run it. This is useful if you are making a change to the -Docker image, are looking to run a bleeding-edge Docker image, or are needing a custom-built Docker image of Data Prepper. +Docker image, are looking to run a bleeding-edge Docker image, or are needing a custom-built Docker image of OpenSearch Data Prepper. ### Building the Docker Image @@ -92,7 +92,7 @@ The repository is `opensearch-data-prepper` and the tag is the current version as defined in [gradle.properties](../gradle.properties). You can run the following command in Linux environments to see -your Data Prepper Docker images: +your OpenSearch Data Prepper Docker images: ``` docker images | grep opensearch-data-prepper @@ -117,18 +117,18 @@ opensearch-data-prepper:2.0.0-SNAPSHOT ``` -## Contributing your Code to Data Prepper +## Contributing your Code to OpenSearch Data Prepper -When you are ready to contribute a change to Data Prepper, please create a GitHub Pull Request (PR). Your PR should target `main`. +When you are ready to contribute a change to OpenSearch Data Prepper, please create a GitHub Pull Request (PR). Your PR should target `main`. -The Data Prepper maintainers will review your PR and merge it once it is approved. +The OpenSearch Data Prepper maintainers will review your PR and merge it once it is approved. Some changes containing bug fixes or security fixes may be eligible for a patch release. If you believe your change should be a patch release, please see [Backporting](#backporting) ### Branches -The Data Prepper maintainers use the `main` branch for the next upcoming release (major or minor). +The OpenSearch Data Prepper maintainers use the `main` branch for the next upcoming release (major or minor). Near the time of the next release, we create a release branch for that upcoming release (e.g. `1.2`). We perform our release builds from this branch. Any patch @@ -137,14 +137,14 @@ releases also build from that release branch. ### Backporting When you create a PR which targets `main` and need this change as a patch to a previous version -of Data Prepper, use the auto backport GitHub Action. All you need to do is add the label +of OpenSearch Data Prepper, use the auto backport GitHub Action. All you need to do is add the label `backport ` to your PR which is targeting `main`. After the PR is merged, the GitHub Action will create a new PR to cherry-pick those changes into the `` branch. -A Data Prepper maintainer will need to approve and merge the backported code into the target branch. +A OpenSearch Data Prepper maintainer will need to approve and merge the backported code into the target branch. The auto-generated PR will be on a branch named `backport/backport--to-`. -Data Prepper supports patch releases only on the latest version (e.g. 2.1) and on the last version +OpenSearch Data Prepper supports patch releases only on the latest version (e.g. 2.1) and on the last version for the previous major release (e.g. 1.4 after 2.0 has been released). These releases are only for bug fixes or security fixes. Please use backports only for bug and security fixes and only targeting candidate releases. You can ask about backporting in your PR or by creating a GitHub @@ -154,7 +154,7 @@ issue to request that a previous change be backported. ### Documentation -Documentation is very important for users of Data Prepper and contributors. We are using the +Documentation is very important for users of OpenSearch Data Prepper and contributors. We are using the following conventions for documentation. 1. Document features in the OpenSearch [documentation-website](https://github.com/opensearch-project/documentation-website). This makes the documentation available at https://opensearch.org/docs/latest/data-prepper. @@ -170,7 +170,7 @@ When you submit a feature PR, please be sure to also submit a new "Documentation Please include in this feature a link to the GitHub issue which has information on the feature. This GitHub issue will often have sample configurations and explanations of the options available to users. Please also provide any additional guidance for the team doing the documentation. -Please include a link to that documentation issue in the PR you created for Data Prepper. +Please include a link to that documentation issue in the PR you created for OpenSearch Data Prepper. You are also welcome to submit a PR directly in the [documentation-website](https://github.com/opensearch-project/documentation-website). @@ -205,7 +205,7 @@ If you are using IntelliJ, you can use the unofficial Checkstyle IDEA plugin. [T ### Dependencies -1. You should first raise an issue in the Data Prepper project if you are interested in adding a new dependency to the core projects. +1. You should first raise an issue in the OpenSearch Data Prepper project if you are interested in adding a new dependency to the core projects. 2. Avoid using dependencies which provide similar functionality to existing dependencies. 1. For example, this project uses Jackson, so do not add Gson 2. If core Java has the function or feature, prefer it over an external library. Example: Guava’s hashcode and equals methods when Java’s Objects class has them. @@ -216,7 +216,7 @@ We have the following categories for tests: * Unit tests - Test a single class in isolation. * Integration tests - Test a large component or set of classes in isolation. -* End-to-end tests - Tests which run an actual Data Prepper. The should generally be in the [`e2e-test`](../e2e-test) project. +* End-to-end tests - Tests which run an actual OpenSearch Data Prepper. The should generally be in the [`e2e-test`](../e2e-test) project. Testing Guidelines: @@ -228,7 +228,7 @@ Testing Guidelines: 4. Each class should have a unit test. 5. Unit test class names should end with Test. 6. Each large component should have an integration test. - 1. A good example is a plugin. Plugins should have their own integration tests which integrate all of the plugin’s classes. However, these tests do not run a full Data Prepper. + 1. A good example is a plugin. Plugins should have their own integration tests which integrate all of the plugin’s classes. However, these tests do not run a full OpenSearch Data Prepper. 7. Integration test class names should end with IT. 8. Test names should indicate what is being tested, if we see a failed test we should be able to look at the test name and have a good idea about what just failed with minimal context about the code being written 1. Two good approaches may be used, depending on what you are testing: @@ -243,7 +243,7 @@ Testing Guidelines: ## CI Builds -Before merging in your PR, the Data Prepper continuous integration (CI) builds must pass. These builds +Before merging in your PR, the OpenSearch Data Prepper continuous integration (CI) builds must pass. These builds run as GitHub Actions. If an Action is failing, please view the log and determine what is causing your commit to fail. If a test