From e8e5025b9577038c6c160b6ea5a1610312b66180 Mon Sep 17 00:00:00 2001 From: Peter Zhu Date: Tue, 31 Oct 2023 14:40:50 -0400 Subject: [PATCH] Onboard jenkins prod docker images to github actions (#924) (#925) * Remove odfe bwc test (#408) (#409) * removed odfe bwc test Signed-off-by: Shenoy Pratik * removed bwc test from workflows Signed-off-by: Shenoy Pratik (cherry picked from commit 556b5fb0a774534ec167dfb9e96f68855296ef75) Co-authored-by: Shenoy Pratik * Remove odfe bwc test (#408) * removed odfe bwc test Signed-off-by: Shenoy Pratik * removed bwc test from workflows Signed-off-by: Shenoy Pratik (cherry picked from commit 556b5fb0a774534ec167dfb9e96f68855296ef75) * Bump version 2.2.0 Signed-off-by: vamsi-amazon * Bump version 2.2.0 (#413) Signed-off-by: vamsi-amazon (cherry picked from commit c91534a67503e6c61ef6e1f53e852bb937f9fc14) Co-authored-by: vamsi-amazon * Release notes for 2.2.0 Signed-off-by: vamsi-amazon (cherry picked from commit f43f7de6eea93f8749eb08ce6cb248275ef78c85) * Release notes for 2.2.0 Signed-off-by: vamsi-amazon (cherry picked from commit f43f7de6eea93f8749eb08ce6cb248275ef78c85) * [2.x] Restrict chromium requests (#435) * Fix regex validation, detect iframe, embed, object tags Signed-off-by: Joshua Li * Disallow redirection to non-localhost urls Signed-off-by: Joshua Li * Disallow connection to non-allowlisted urls Signed-off-by: Joshua Li * Disable JIT Signed-off-by: Joshua Li * Fix workflow Signed-off-by: Joshua Li * Try to fix CI Signed-off-by: Joshua Li * Fix localstorage logic Signed-off-by: Joshua Li Signed-off-by: Joshua Li * [2.2] Restrict chromium requests (#431) * Fix regex validation, detect iframe, embed, object tags Signed-off-by: Joshua Li * Disallow redirection to non-localhost urls Signed-off-by: Joshua Li * Disallow connection to non-allowlisted urls Signed-off-by: Joshua Li * Disable JIT Signed-off-by: Joshua Li * Fix workflow Signed-off-by: Joshua Li * Try to fix CI Signed-off-by: Joshua Li * Fix localstorage logic Signed-off-by: Joshua Li Signed-off-by: Joshua Li * Increment version to 2.2.1-SNAPSHOT Signed-off-by: opensearch-ci-bot * Increment version to 2.3.0-SNAPSHOT Signed-off-by: opensearch-ci-bot * Added release notes for 2.2.1 release Signed-off-by: vamsi-amazon * Merge pull request #459 from mengweieric/release-notes-2.3.0 Release notes for 2.3.0 (cherry picked from commit b0c564338323056d33fddf30e70a5b0e59b69c6a) * [2.x] Upgrade puppeteer (#489) * upgrade puppeteer & change report timeout Signed-off-by: Rupal Mahajan * Change timeout Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Bump verison 2.4.0 (#499) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * add group = org.opensearch.plugin (#506) (#507) Signed-off-by: prudhvigodithi Signed-off-by: prudhvigodithi (cherry picked from commit b25984bbd9dad06e331a9b33330d6b31894f8651) Co-authored-by: Prudhvi Godithi * feat: enable windows and macos build (#504) (#511) Signed-off-by: Derek Ho * Update version for jsdom, terser and jsoup (#515) Signed-off-by: Rupal Mahajan * Fix snakeyaml vulnerability issue by disabling detekt (#517) Signed-off-by: Rupal Mahajan * Update ktlint version (#519) Signed-off-by: Rupal Mahajan * Upgrade minimatch, moment (#513) Signed-off-by: Rupal Mahajan * Revert detekt changes with upgraded version 1.21.0 (#527) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Add loader-utils to resolutions (#525) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Nit: add resolutions at once place (#529) (#530) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan (cherry picked from commit 0dada242c5ab0fcc22e6d330eeaaa71bb6796b23) Co-authored-by: Rupal Mahajan * add release notes for 2.4.0.0 (#532) (#533) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan (cherry picked from commit a1c4839c10f5737dd73b6ce3e1781d4356d1de2e) Co-authored-by: Rupal Mahajan * Bump loader-utils from 2.0.3 to 2.0.4 in /dashboards-reports (#539) (#541) Bumps [loader-utils](https://github.com/webpack/loader-utils) from 2.0.3 to 2.0.4. - [Release notes](https://github.com/webpack/loader-utils/releases) - [Changelog](https://github.com/webpack/loader-utils/blob/v2.0.4/CHANGELOG.md) - [Commits](https://github.com/webpack/loader-utils/compare/v2.0.3...v2.0.4) --- updated-dependencies: - dependency-name: loader-utils dependency-type: indirect ... Signed-off-by: dependabot[bot] Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit f250deb48333bb49a2cc202896eec74c2ee9ea36) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update qs 6.5.3 (#545) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Fix metrics tests and ClassNotFoundException when calling stats API (#546) Signed-off-by: Joshua Li * Use advanced settings for leading wildcards in query for csv reports (#549) * Fetch allowLeadingWildcards from config Signed-off-by: Rupal Mahajan * Fix tests Signed-off-by: Rupal Mahajan * nit Signed-off-by: Rupal Mahajan * add default value for allowLeadingWildCards Co-authored-by: Joshua Li Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan Co-authored-by: Joshua Li * Increment version to 2.5.0-SNAPSHOT (#528) Signed-off-by: opensearch-ci-bot Signed-off-by: opensearch-ci-bot Co-authored-by: opensearch-ci-bot * [1.x] Upgrade decode-uri-component (#558) (#560) * upgrade loader-utils for CVE-2022-37601 Signed-off-by: Rupal Mahajan * Update async for CVE-2021-43138 Signed-off-by: Rupal Mahajan * Fix cross-fetch for CVE-2022-1365 Signed-off-by: Rupal Mahajan * Fix for CVE-2022-37599, CVE-2022-37603 Signed-off-by: Rupal Mahajan * Update terser for CVE-2022-25858 Signed-off-by: Rupal Mahajan * Update minimatch for CVE-2022-3517 Signed-off-by: Rupal Mahajan * Update moment for CVE-2022-24785 Signed-off-by: Rupal Mahajan * Update jsdom for CVE-2021-20066 Signed-off-by: Rupal Mahajan * Update execa for GMS-2020-2 Signed-off-by: Rupal Mahajan * Update qs for CVE-2022-24999 Signed-off-by: Rupal Mahajan * Update moment for CVE-2022-31129 Signed-off-by: Rupal Mahajan * Update decode-uri-component for CVE-2022-38900 Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan (cherry picked from commit d3f36b512fe2982a2bab4d4086ad4ba6a597631a) Co-authored-by: Rupal Mahajan * Fix windows and macos CI (#569) (#571) * Update windows & mac CI for reo name change Signed-off-by: Rupal Mahajan * Fix linux build Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan (cherry picked from commit 9ce868480beff4cf21746609a5415efa225e18b8) Co-authored-by: Rupal Mahajan * Add ERR_ADDRESS_INVALID in Troubleshooting (#574) * Add workaround for ERR_ADDRESS_INVALID in doc Signed-off-by: Rupal Mahajan * nit Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Add release notes for 2.4.1 (#580) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Remove jackson-databind and jackson-annotations (#587) * Remove jackson-databind and jackson-annotations dependencies Signed-off-by: Rupal Mahajan * resolve dompurify conflicting dependencies Signed-off-by: Rupal Mahajan * fix build Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Use front-end report generation instead of chromium (#586) * Increment version to 2.4.1-SNAPSHOT (#540) Signed-off-by: opensearch-ci-bot Signed-off-by: opensearch-ci-bot Co-authored-by: opensearch-ci-bot * --wip-- Signed-off-by: Joshua Li * Add initial implementation of client reporting generation Signed-off-by: Joshua Li * Fix url with basepath Signed-off-by: Joshua Li * Update header footer height Signed-off-by: Joshua Li * Update dialog text to not close dialog Signed-off-by: Joshua Li * Remove console.log Signed-off-by: Joshua Li * Remove unused components Signed-off-by: Joshua Li * Remove chromium references Signed-off-by: Joshua Li * Add report generation error handling Signed-off-by: Joshua Li * Minor refactors Signed-off-by: Joshua Li * Add postinstall patch to support safari for html2canvas Signed-off-by: Joshua Li * Add dompurify Signed-off-by: Joshua Li * Fix build error Signed-off-by: Joshua Li * Remove chromium from CI Signed-off-by: Joshua Li * Update CI artifact name Signed-off-by: Joshua Li Signed-off-by: opensearch-ci-bot Signed-off-by: Joshua Li Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot * Remove front end code (#620) * Remove frontend code Signed-off-by: Rupal Mahajan * Move one level up Signed-off-by: Rupal Mahajan * Update workflow file Signed-off-by: Rupal Mahajan * nit: remove dashboard setup from readme Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Remove unnecessary scripts after repo split (#622) Signed-off-by: Joshua Li * Include integration tests in windows workflow (#624) Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Add release notes for 2.5.0.0 (#628) * Add release notes for 2.5.0.0 Signed-off-by: Rupal Mahajan * Fix version in release notes Signed-off-by: Rupal Mahajan Signed-off-by: Rupal Mahajan * Increment version to 2.6.0 (#636) Signed-off-by: Rupal Mahajan * Updated MAINTAINERS.md to match recommended opensearch-project format. (#619) (#635) Signed-off-by: dblock Signed-off-by: dblock (cherry picked from commit 8e1cdadcc95ebae8122b5d07c4f5354addbdc77d) Co-authored-by: Daniel (dB.) Doubrovkine * [Backport 2.x] Add jackson-databind and jackson-annotations (#652) * Add jackson-databind and jackson-annotations (#650) Signed-off-by: Rupal Mahajan (cherry picked from commit cc34ae81a730b954062c61d643c3587c63ebbfd0) * Resolve mockito-core conflict Signed-off-by: Rupal Mahajan --------- Signed-off-by: Rupal Mahajan Co-authored-by: Rupal Mahajan * Add publish to maven workflow (#651) (#655) Signed-off-by: Rupal Mahajan (cherry picked from commit 1c7a917258eee69f3906223eee13dc7278bcff5d) Co-authored-by: Rupal Mahajan * fix maven publish workflow (#665) (#666) * fix maven publish workflow Signed-off-by: Derek Ho * remove bracket Signed-off-by: Derek Ho * fix branch names Signed-off-by: Derek Ho * stringify 1.3 Signed-off-by: Derek Ho --------- Signed-off-by: Derek Ho (cherry picked from commit 354f44a859c60e979036417a4e1a9f9228451520) Co-authored-by: Derek Ho * Increment version to 2.7.0-SNAPSHOT (#657) Signed-off-by: opensearch-ci-bot Co-authored-by: opensearch-ci-bot * Merge pull request #656 from rupal-bq/release-notes-2.6.0 (#659) Add release notes for 2.6.0 (cherry picked from commit 6c1b03ed045cd3d2bd8d0b2d14e49ceb65bbd76c) Signed-off-by: Derek Ho Co-authored-by: Eric Wei * Bump snakeyaml to 2.0 (#674) Signed-off-by: Joshua Li (cherry picked from commit b7923960e0f6d8f61cbfa3dddd4c283c07155e62) * upgrade mockito-core (#678) (#679) Signed-off-by: Rupal Mahajan (cherry picked from commit 14ecd4433aa7b40995839240199575c2939bfe4e) Co-authored-by: Rupal Mahajan * [Backport 2.x] Fix ci failures (#681) * Fix ci failures (#662) Signed-off-by: Yury-Fridlyand (cherry picked from commit 07956f7d672e4e9a7a0d71a4f32599b0a87210b4) * Pull Job scheduler from maven instead of jenkins (#663) * use maven Signed-off-by: Derek Ho * bwc version to 2.7.0 bump Signed-off-by: Derek Ho * hard code zips Signed-off-by: Derek Ho --------- Signed-off-by: Derek Ho Signed-off-by: Rupal Mahajan --------- Signed-off-by: Derek Ho Signed-off-by: Rupal Mahajan Co-authored-by: Yury-Fridlyand Co-authored-by: Derek Ho * Add release notes for 2.7 (#684) (#686) Signed-off-by: Rupal Mahajan (cherry picked from commit 8fafd728aafcbc27fbef87243e4537e41f22b3dd) Co-authored-by: Rupal Mahajan * Update json version to 20230227 (#692) Signed-off-by: Rupal Mahajan * Update Gradle Wrapper to 7.6.1 (#689) (#695) Signed-off-by: Daniel Widdis (cherry picked from commit ebbb54d9d80538ae782ec7972d1b2a1988dea858) Co-authored-by: Daniel Widdis * Increment version to 2.8.0-SNAPSHOT (#688) Signed-off-by: opensearch-ci-bot Co-authored-by: opensearch-ci-bot * Add release notes for 2.8 (#705) (#706) Signed-off-by: Joshua Li (cherry picked from commit f27cdd45c16c39366e7f0666a45bc363e79024b1) Co-authored-by: Joshua Li * Removing guava dependency to fix jarhell (#709) (#710) * removing guava to fix jarhell Signed-off-by: Shenoy Pratik * update release notes Signed-off-by: Shenoy Pratik --------- Signed-off-by: Shenoy Pratik (cherry picked from commit d7ff8cd6b6c97012c53de718835a2fd4985479bf) Co-authored-by: Shenoy Pratik * Increment version to 2.9.0-SNAPSHOT Signed-off-by: opensearch-ci-bot * Add 2.9 release notes (#723) (#724) Signed-off-by: Kavitha Conjeevaram Mohan (cherry picked from commit 79a385b831bbb03dd3b5c770f538f046bdbf16f8) Co-authored-by: Kavitha Conjeevaram Mohan * fix ci Signed-off-by: Derek Ho * update kotlin version Signed-off-by: Derek Ho * update build.gradle Signed-off-by: Derek Ho * ktlint Signed-off-by: Derek Ho * version bump Signed-off-by: Derek Ho * update workflow Signed-off-by: Derek Ho * fix imports in tests Signed-off-by: Derek Ho * update import from upstream breaking changes Signed-off-by: Derek Ho * Merge pull request #748 from derek-ho/insights fix from upstream import changes (cherry picked from commit c254aad4260c180dad2a8862359df8e2de1ae7f5) Signed-off-by: Derek Ho * Update PluginBaseAction.kt Remove duplicate import Signed-off-by: Derek Ho * Update backport CI, add PR merged condition (#750) Signed-off-by: Shenoy Pratik * Add 2.10 release notes (#758) (#759) (cherry picked from commit 6a9363138ddf039e083c0994e9614319f780b2d5) Signed-off-by: Rupal Mahajan Signed-off-by: github-actions[bot] Co-authored-by: github-actions[bot] * Increment version to 2.11.0-SNAPSHOT (#756) Signed-off-by: opensearch-ci-bot Co-authored-by: opensearch-ci-bot * Updates demo certs used in integ tests (#755) (#757) (cherry picked from commit 3d17d50323898aed94ace8ff8fb16280928f5010) Signed-off-by: Darshit Chanpura Signed-off-by: github-actions[bot] Co-authored-by: github-actions[bot] * Increment version to 2.12.0-SNAPSHOT (#871) Signed-off-by: opensearch-ci-bot Co-authored-by: opensearch-ci-bot * Upgrade JSON to 20231013 to fix CVE-2023-5072 (#912) (#914) (cherry picked from commit b68990fed59814832800affed06e5aa4439f0918) Signed-off-by: Derek Ho Signed-off-by: github-actions[bot] Co-authored-by: github-actions[bot] * Onboard jenkins prod docker images to github actions (#924) Signed-off-by: Peter Zhu --------- Signed-off-by: vamsi-amazon Signed-off-by: Joshua Li Signed-off-by: opensearch-ci-bot Signed-off-by: Eric Wei Signed-off-by: Rupal Mahajan Signed-off-by: Derek Ho Signed-off-by: Shenoy Pratik Signed-off-by: github-actions[bot] Signed-off-by: Darshit Chanpura Signed-off-by: Peter Zhu Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: Shenoy Pratik Co-authored-by: vamsi-amazon Co-authored-by: Joshua Li Co-authored-by: opensearch-ci-bot Co-authored-by: Prudhvi Godithi Co-authored-by: Eric Wei Co-authored-by: Rupal Co-authored-by: Derek Ho Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Daniel (dB.) Doubrovkine Co-authored-by: Derek Ho Co-authored-by: Yury-Fridlyand Co-authored-by: Daniel Widdis Co-authored-by: Kavitha Conjeevaram Mohan Co-authored-by: github-actions[bot] --- .github/workflows/dco.yml | 18 ------ .../reports-scheduler-release-workflow.yml | 55 ------------------- ...orts-scheduler-test-and-build-workflow.yml | 22 +++++++- 3 files changed, 21 insertions(+), 74 deletions(-) delete mode 100644 .github/workflows/dco.yml delete mode 100644 .github/workflows/reports-scheduler-release-workflow.yml diff --git a/.github/workflows/dco.yml b/.github/workflows/dco.yml deleted file mode 100644 index cf30ea89..00000000 --- a/.github/workflows/dco.yml +++ /dev/null @@ -1,18 +0,0 @@ -name: Developer Certificate of Origin Check - -on: [pull_request] - -jobs: - check: - runs-on: ubuntu-latest - - steps: - - name: Get PR Commits - id: 'get-pr-commits' - uses: tim-actions/get-pr-commits@v1.1.0 - with: - token: ${{ secrets.GITHUB_TOKEN }} - - name: DCO Check - uses: tim-actions/dco@v1.1.0 - with: - commits: ${{ steps.get-pr-commits.outputs.commits }} diff --git a/.github/workflows/reports-scheduler-release-workflow.yml b/.github/workflows/reports-scheduler-release-workflow.yml deleted file mode 100644 index e418c58f..00000000 --- a/.github/workflows/reports-scheduler-release-workflow.yml +++ /dev/null @@ -1,55 +0,0 @@ -name: Release Reports Scheduler Artifacts -# This workflow is triggered on creating tags to main or an opensearch release branch -on: - push: - tags: - - "v*" - -jobs: - build: - runs-on: ubuntu-latest - - steps: - - name: Configure AWS Credentials - uses: aws-actions/configure-aws-credentials@v1 - with: - aws-access-key-id: ${{ secrets.AWS_STAGING_ACCESS_KEY_ID }} - aws-secret-access-key: ${{ secrets.AWS_STAGING_SECRET_ACCESS_KEY }} - aws-region: us-east-1 - - - name: Checkout Plugin - uses: actions/checkout@v1 - - - name: Set up JDK 1.17 - uses: actions/setup-java@v1 - with: - java-version: 1.17 - - - name: Run build - run: | - cd reports-scheduler - ./gradlew build buildDeb buildRpm --no-daemon --refresh-dependencies -Dbuild.snapshot=false - - - name: Upload to S3 - shell: bash - run: | - cd reports-scheduler - zip=`ls build/distributions/*.zip` - rpm=`ls build/distributions/*.rpm` - deb=`ls build/distributions/*.deb` - - # Inject the build number before the suffix - zip_outfile=`basename ${zip%.zip}-build-${GITHUB_RUN_NUMBER}.zip` - rpm_outfile=`basename ${rpm%.rpm}-build-${GITHUB_RUN_NUMBER}.rpm` - deb_outfile=`basename ${deb%.deb}-build-${GITHUB_RUN_NUMBER}.deb` - - s3_prefix="s3://staging.artifacts.opendistroforelasticsearch.amazon.com/snapshots/elasticsearch-plugins/reports-scheduler/" - - echo "Copying ${zip} to ${s3_prefix}${zip_outfile}" - aws s3 cp --quiet $zip ${s3_prefix}${zip_outfile} - - echo "Copying ${rpm} to ${s3_prefix}${rpm_outfile}" - aws s3 cp --quiet $rpm ${s3_prefix}${rpm_outfile} - - echo "Copying ${deb} to ${s3_prefix}${deb_outfile}" - aws s3 cp --quiet $deb ${s3_prefix}${deb_outfile} diff --git a/.github/workflows/reports-scheduler-test-and-build-workflow.yml b/.github/workflows/reports-scheduler-test-and-build-workflow.yml index 00d4bf45..327018bb 100644 --- a/.github/workflows/reports-scheduler-test-and-build-workflow.yml +++ b/.github/workflows/reports-scheduler-test-and-build-workflow.yml @@ -3,13 +3,25 @@ name: Test and Build Reports Scheduler on: [push, pull_request] jobs: + Get-CI-Image-Tag: + uses: opensearch-project/opensearch-build/.github/workflows/get-ci-image-tag.yml@main + with: + product: opensearch + linux-build: + needs: Get-CI-Image-Tag strategy: matrix: java: - 11 - 17 runs-on: ubuntu-latest + container: + # using the same image which is used by opensearch-build team to build the OpenSearch Distribution + # this image tag is subject to change as more dependencies and updates will arrive over time + image: ${{ needs.Get-CI-Image-Tag.outputs.ci-image-version-linux }} + # need to switch to root so that github actions can install runner binary on container without permission issues. + options: --user root steps: - name: Set up JDK ${{ matrix.java }} @@ -20,9 +32,16 @@ jobs: - name: Checkout Reports Scheduler uses: actions/checkout@v2 + - name: RunBackwards Compatibility Tests + run: | + echo "Running backwards compatibility tests ..." + chown -R 1000:1000 `pwd` + su `id -un 1000` -c "./gradlew bwcTestSuite" + - name: Build with Gradle run: | - ./gradlew build + chown -R 1000:1000 `pwd` + su `id -un 1000` -c "./gradlew build" - name: Upload coverage uses: codecov/codecov-action@v1 @@ -35,6 +54,7 @@ jobs: run: | mkdir -p reports-scheduler-builds cp -r ./build/distributions/*.zip reports-scheduler-builds/ + chown -R 1000:1000 `pwd` - name: Upload Artifacts uses: actions/upload-artifact@v1