Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2.0-pick-26525](workflow) Fix security issues with pull_request_target #26525 #29289

Merged
merged 2 commits into from
Jan 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/actions/action-sh-checker
Submodule action-sh-checker deleted from 76ab0b
1 change: 0 additions & 1 deletion .github/actions/clang-format-lint-action
Submodule clang-format-lint-action deleted from 6adbe1
1 change: 0 additions & 1 deletion .github/actions/clang-tidy-review
Submodule clang-tidy-review deleted from 2c55ef
13 changes: 0 additions & 13 deletions .github/actions/patches/action-sh-checker.patch

This file was deleted.

25 changes: 24 additions & 1 deletion .github/workflows/clang-format.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,21 @@ jobs:
uses: actions/checkout@v3
with:
persist-credentials: false
submodules: recursive

- name: Checkout ${{ github.ref }} ( ${{ github.event.pull_request.head.sha }} )
if: ${{ github.event_name == 'pull_request_target' }}
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Checkout paths-filter
run: |
rm -rf ./.github/actions/paths-filter
git clone https://github.com/dorny/paths-filter .github/actions/paths-filter

pushd .github/actions/paths-filter &>/dev/null
git checkout 4512585405083f25c027a35db413c2b3b9006d50
popd &>/dev/null

- name: Paths filter
uses: ./.github/actions/paths-filter
Expand All @@ -41,6 +55,15 @@ jobs:
- 'be/src/**'
- 'be/test/**'

- name: Checkout clang-format-lint-action
run: |
rm -rf ./.github/actions/clang-format-lint-action
git clone https://github.com/DoozyX/clang-format-lint-action .github/actions/clang-format-lint-action

pushd .github/actions/clang-format-lint-action &>/dev/null
git checkout 6adbe14579e5b8e19eb3e31e5ff2479f3bd302c7
popd &>/dev/null

- name: "Format it!"
if: ${{ steps.filter.outputs.be_changes == 'true' }}
uses: ./.github/actions/clang-format-lint-action
Expand Down
98 changes: 73 additions & 25 deletions .github/workflows/code-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,21 +27,22 @@ jobs:
- name: Checkout ${{ github.ref }} ( ${{ github.sha }} )
if: ${{ github.event_name != 'pull_request_target' }}
uses: actions/checkout@v3
with:
submodules: recursive

- name: Checkout ${{ github.ref }} ( ${{ github.event.pull_request.head.sha }} )
if: ${{ github.event_name == 'pull_request_target' }}
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
submodules: recursive

- name: Patch
- name: Checkout action-sh-checker
run: |
pushd .github/actions/action-sh-checker >/dev/null
git apply ../patches/action-sh-checker.patch
popd >/dev/null
rm -rf ./.github/actions/action-sh-checker
git clone https://github.com/luizm/action-sh-checker .github/actions/action-sh-checker

pushd .github/actions/action-sh-checker &>/dev/null
git checkout 76ab0b22e1f194e4a582edc7969df6485c4e9246
sed -i 's/\[ "$GITHUB_EVENT_NAME" == "pull_request" \]/\[\[ "$GITHUB_EVENT_NAME" == "pull_request" || "$GITHUB_EVENT_NAME" == "pull_request_target" \]\]/' entrypoint.sh
popd &>/dev/null

- name: Run ShellCheck
uses: ./.github/actions/action-sh-checker
Expand All @@ -51,16 +52,27 @@ jobs:
sh_checker_comment: true
sh_checker_exclude: .git .github ^docker ^thirdparty/src ^thirdparty/installed ^ui ^docs/node_modules ^tools/clickbench-tools ^extension ^output ^fs_brokers/apache_hdfs_broker/output (^|.*/)Dockerfile$ ^be/src/apache-orc ^be/src/clucene ^pytest

clang-tidy:
name: "Clang Tidy"
preparation:
name: "Clang Tidy Preparation"
if: ${{ github.event_name == 'pull_request_target' }}
runs-on: ubuntu-22.04
permissions: read-all
outputs:
should_check: ${{ steps.generate.outputs.should_check }}
steps:
- name: Checkout ${{ github.ref }} ( ${{ github.event.pull_request.head.sha }} )
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
submodules: recursive

- name: Checkout paths-filter
run: |
rm -rf ./.github/actions/paths-filter
git clone https://github.com/dorny/paths-filter .github/actions/paths-filter

pushd .github/actions/paths-filter &>/dev/null
git checkout 4512585405083f25c027a35db413c2b3b9006d50
popd &>/dev/null

- name: Paths Filter
uses: ./.github/actions/paths-filter
Expand All @@ -73,28 +85,64 @@ jobs:
- 'gensrc/thrift/**'

- name: Generate compile_commands.json
if: ${{ steps.filter.outputs.be_changes == 'true' }}
id: generate
run: |
export DEFAULT_DIR='/opt/doris'
if [[ "${{ steps.filter.outputs.be_changes }}" == 'true' ]]; then
export DEFAULT_DIR='/opt/doris'

mkdir "${DEFAULT_DIR}"
wget https://github.com/amosbird/ldb_toolchain_gen/releases/download/v0.18/ldb_toolchain_gen.sh \
-q -O /tmp/ldb_toolchain_gen.sh
bash /tmp/ldb_toolchain_gen.sh "${DEFAULT_DIR}/ldb-toolchain"
mkdir "${DEFAULT_DIR}"
wget https://github.com/amosbird/ldb_toolchain_gen/releases/download/v0.18/ldb_toolchain_gen.sh \
-q -O /tmp/ldb_toolchain_gen.sh
bash /tmp/ldb_toolchain_gen.sh "${DEFAULT_DIR}/ldb-toolchain"

sudo DEBIAN_FRONTEND=noninteractive apt install --yes tzdata byacc
sudo DEBIAN_FRONTEND=noninteractive apt install --yes tzdata byacc

pushd thirdparty
curl -L https://github.com/apache/doris-thirdparty/releases/download/automation/doris-thirdparty-prebuilt-linux-x86_64.tar.xz \
-o doris-thirdparty-prebuilt-linux-x86_64.tar.xz
tar -xvf doris-thirdparty-prebuilt-linux-x86_64.tar.xz
popd
pushd thirdparty
curl -L https://github.com/apache/doris-thirdparty/releases/download/automation/doris-thirdparty-prebuilt-linux-x86_64.tar.xz \
-o doris-thirdparty-prebuilt-linux-x86_64.tar.xz
tar -xvf doris-thirdparty-prebuilt-linux-x86_64.tar.xz
popd

export PATH="${DEFAULT_DIR}/ldb-toolchain/bin/:$(pwd)/thirdparty/installed/bin/:${PATH}"
DISABLE_JAVA_UDF=ON DORIS_TOOLCHAIN=clang OUTPUT_BE_BINARY=0 ./build.sh --be
export PATH="${DEFAULT_DIR}/ldb-toolchain/bin/:$(pwd)/thirdparty/installed/bin/:${PATH}"
DISABLE_JAVA_UDF=ON DORIS_TOOLCHAIN=clang ENABLE_PCH=OFF OUTPUT_BE_BINARY=0 ./build.sh --be
fi

- name: Run clang-tidy review
echo "should_check=${{ steps.filter.outputs.be_changes }}" >>${GITHUB_OUTPUT}

- name: Upload
uses: actions/upload-artifact@v3
if: ${{ steps.filter.outputs.be_changes == 'true' }}
with:
name: compile_commands
path: ./be/build_Release/compile_commands.json

clang-tidy:
name: "Clang Tidy"
needs: preparation
if: ${{ needs.preparation.outputs.should_check == 'true' }}
runs-on: ubuntu-22.04
steps:
- name: Checkout ${{ github.ref }} ( ${{ github.event.pull_request.head.sha }} )
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Download
uses: actions/download-artifact@v3
with:
name: compile_commands
path: ./be/build_Release

- name: Checkout clang-tidy review
run: |
rm -rf ./.github/actions/clang-tidy-review
git clone https://github.com/ZedThree/clang-tidy-review .github/actions/clang-tidy-review

pushd .github/actions/clang-tidy-review &>/dev/null
git checkout 2c55ef8cfc9acb3715d433e58aea086dcec9b206
popd &>/dev/null

- name: Run clang-tidy review
uses: ./.github/actions/clang-tidy-review
id: review
with:
Expand Down
9 changes: 0 additions & 9 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,6 @@
[submodule ".github/actions/get-workflow-origin"]
path = .github/actions/get-workflow-origin
url = https://github.com/potiuk/get-workflow-origin.git
[submodule ".github/actions/clang-format-lint-action"]
path = .github/actions/clang-format-lint-action
url = https://github.com/DoozyX/clang-format-lint-action.git
[submodule ".github/actions/setup-maven"]
path = .github/actions/setup-maven
url = https://github.com/stCarolas/setup-maven.git
Expand All @@ -19,12 +16,6 @@
[submodule ".github/actions/ccache-action"]
path = .github/actions/ccache-action
url = https://github.com/hendrikmuhs/ccache-action
[submodule ".github/actions/action-sh-checker"]
path = .github/actions/action-sh-checker
url = https://github.com/luizm/action-sh-checker
[submodule ".github/actions/clang-tidy-review"]
path = .github/actions/clang-tidy-review
url = https://github.com/ZedThree/clang-tidy-review.git
[submodule "be/src/apache-orc"]
path = be/src/apache-orc
url = https://github.com/apache/doris-thirdparty.git
Expand Down
Loading