Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to Github Actions #3152

Merged
merged 36 commits into from
Mar 20, 2020
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
bdc72e7
WIP: Trying to make a new branch
francislavoie Mar 17, 2020
c67b7cc
Create fuzzing.yml
francislavoie Mar 17, 2020
cb6b638
Update ci.yml
francislavoie Mar 17, 2020
403b4c3
Try using reviewdog for golangci-lint
francislavoie Mar 17, 2020
4da0b20
Only run lint on ubuntu
francislavoie Mar 17, 2020
1d02adc
Whoops, wrong matrix variable
francislavoie Mar 17, 2020
92c3803
Let's try just ubuntu for the moment
francislavoie Mar 17, 2020
34ba43e
Remove integration tests
francislavoie Mar 20, 2020
4cedbfd
Let's see what the tree looks like (where's the binary)
francislavoie Mar 20, 2020
029e190
Let's plant a tree
francislavoie Mar 20, 2020
19196bb
Let's look at another tree
francislavoie Mar 20, 2020
9e7e4f4
Burn the tree
francislavoie Mar 20, 2020
0e8e928
Let's build in the right dir
francislavoie Mar 20, 2020
5750efd
Turn on publishing artifacts
francislavoie Mar 20, 2020
b3a6dd1
Add gobin to path
francislavoie Mar 20, 2020
a037377
Try running golangci-lint earlier
francislavoie Mar 20, 2020
022d5b0
Try running golangci-lint on its own, with checkout@v1
francislavoie Mar 20, 2020
3123bb5
Try moving golangci-lint back into ci.yml as a separate job
francislavoie Mar 20, 2020
8ead759
Turn off azure-pipelines
francislavoie Mar 20, 2020
7d4f6e8
Remove the redundant name, see how it looks
francislavoie Mar 20, 2020
a3e0de3
Trim down the naming some more
francislavoie Mar 20, 2020
db49dd3
Turn on windows and mac
francislavoie Mar 20, 2020
22b31a9
Try to fix windows build, cleanup
francislavoie Mar 20, 2020
89e1126
Try to fix strange failure on windows
francislavoie Mar 20, 2020
2664ee7
Print our the coerce reason
francislavoie Mar 20, 2020
989f19a
Apparently $? is 'True' on Windows, not 1 or 0
francislavoie Mar 20, 2020
e4b2eb8
Try setting CGO_ENABLED as an env in yml
francislavoie Mar 20, 2020
b0e0280
Try enabling/fixing the fuzzer
francislavoie Mar 20, 2020
e0748d3
Print out github event to check, fix step name
francislavoie Mar 20, 2020
a7ccf69
Fuzzer needs the code
francislavoie Mar 20, 2020
78c750c
Add GOBIN to PATH for fuzzer
francislavoie Mar 20, 2020
89d53af
Comment out fork condition, left in-case we want it again
francislavoie Mar 20, 2020
bbcbfc3
Remove obsolete comment
francislavoie Mar 20, 2020
90b3891
Comment out the coverage/test conversions for now
francislavoie Mar 20, 2020
d81d602
Set continue-on-error: true for fuzzer, it runs out of mem
francislavoie Mar 20, 2020
ea4bdd7
Add some clarification to the retained commented sections
francislavoie Mar 20, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Used as inspiration: https://github.com/mvdan/github-actions-golang

name: Cross-Platform

on:
push:
branches:
- v2
pull_request:
branches:
- v2

jobs:
test:
strategy:
# Default is true, cancels jobs for other platforms if one fails
fail-fast: false
matrix:
os: [ ubuntu-latest, macos-latest, windows-latest ]
go-version: [ 1.14.x ]
runs-on: ${{ matrix.os }}

steps:
- name: Install Go
uses: actions/setup-go@v1
with:
go-version: ${{ matrix.go-version }}

- name: Checkout code
uses: actions/checkout@v2

# - name: Install test and coverage analysis tools
# run: |
# go get github.com/axw/gocov/gocov
# go get github.com/AlekSi/gocov-xml
# go get -u github.com/jstemmer/go-junit-report
# echo "::add-path::$(go env GOPATH)/bin"
mholt marked this conversation as resolved.
Show resolved Hide resolved

- name: Print Go version and environment
run: |
printf "Using go at: $(which go)\n"
printf "Go version: $(go version)\n"
printf "\n\nGo environment:\n\n"
go env
printf "\n\nSystem environment:\n\n"
env

- name: Get dependencies
run: |
go get -v -t -d ./...
# mkdir test-results

- name: Build Caddy
working-directory: ./cmd/caddy
env:
CGO_ENABLED: 0
run: |
go build -trimpath -a -ldflags="-w -s" -v

- name: Publish Build Artifact (Windows)
if: matrix.os == 'windows-latest'
uses: actions/upload-artifact@v1
with:
name: caddy_v2_${{ matrix.os }}
path: ./cmd/caddy/caddy.exe

- name: Publish Build Artifact (Linux/Mac)
if: matrix.os != 'windows-latest'
uses: actions/upload-artifact@v1
with:
name: caddy_v2_${{ matrix.os }}
path: ./cmd/caddy/caddy

# For info about set-output, see https://stackoverflow.com/questions/57850553/github-actions-check-steps-status
- name: Run tests
# id: step_test
# continue-on-error: true
run: |
# (go test -v -coverprofile=cover-profile.out -race ./... 2>&1) > test-results/test-result.out
go test -v -coverprofile="cover-profile.out" -race ./...
echo "::set-output name=status::$?"

# - name: Prepare coverage reports
# run: |
# mkdir coverage
# gocov convert cover-profile.out > coverage/coverage.json
# # Because Windows doesn't work with input redirection like *nix, but output redirection works.
# (cat ./coverage/coverage.json | gocov-xml) > coverage/coverage.xml

# TODO publish coverage/lint/test results ?

# To return the correct result even though we set 'continue-on-error: true'
# - name: Coerce correct build result (Windows)
# if: matrix.os == 'windows-latest' && steps.step_test.outputs.status != 'True'
# run: |
# echo "step_test ${{ steps.step_test.outputs.status }}\n"
# exit 1

# - name: Coerce correct build result (Linux/Mac)
# if: matrix.os != 'windows-latest' && steps.step_test.outputs.status != 0
# run: |
# echo "step_test ${{ steps.step_test.outputs.status }}\n"
# exit 1

golangci-lint:
name: runner / golangci-lint
runs-on: ubuntu-latest
steps:
- name: Checkout code into the Go module directory
uses: actions/checkout@v2

- name: Run golangci-lint
uses: reviewdog/action-golangci-lint@v1
# uses: docker://reviewdog/action-golangci-lint:v1 # pre-build docker image
with:
github_token: ${{ secrets.github_token }}
83 changes: 83 additions & 0 deletions .github/workflows/fuzzing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: Fuzzing

on:
# Regression testing
push:
branches:
- v2
pull_request:
branches:
- v2

# Daily midnight fuzzing
schedule:
francislavoie marked this conversation as resolved.
Show resolved Hide resolved
- cron: '0 0 * * *'

jobs:
fuzzing:
name: Fuzzing

strategy:
matrix:
os: [ ubuntu-latest ]
go-version: [ 1.14.x ]
runs-on: ${{ matrix.os }}

steps:
- name: Install Go
uses: actions/setup-go@v1
with:
go-version: ${{ matrix.go-version }}

- name: Checkout code
uses: actions/checkout@v2

- name: Download go-fuzz tools and the Fuzzit CLI, move Fuzzit CLI to GOBIN
# if: github.repository == 'caddyserver/caddy'
run: |
# Install Clang-7.0 because other versions seem to be missing the file libclang_rt.fuzzer-x86_64.a
sudo add-apt-repository "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-7 main"
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
sudo apt update && sudo apt install -y clang-7 lldb-7 lld-7

go get -v github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build
wget -q -O fuzzit https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.77/fuzzit_Linux_x86_64
chmod a+x fuzzit
mv fuzzit $(go env GOPATH)/bin
echo "::add-path::$(go env GOPATH)/bin"

- name: Generate fuzzers & submit them to Fuzzit
continue-on-error: true
env:
FUZZIT_API_KEY: ${{ secrets.FUZZIT_API_KEY }}
run: |
declare -A fuzzers_funcs=(\
["./caddyconfig/httpcaddyfile/addresses_fuzz.go"]="FuzzParseAddress" \
["./caddyconfig/caddyfile/parse_fuzz.go"]="FuzzParseCaddyfile" \
["./listeners_fuzz.go"]="FuzzParseNetworkAddress" \
["./replacer_fuzz.go"]="FuzzReplacer" \
)

declare -A fuzzers_targets=(\
["./caddyconfig/httpcaddyfile/addresses_fuzz.go"]="parse-address" \
["./caddyconfig/caddyfile/parse_fuzz.go"]="parse-caddyfile" \
["./listeners_fuzz.go"]="parse-network-address" \
["./replacer_fuzz.go"]="replacer" \
)

fuzz_type="local-regression"
if [[ ${{ github.event_name }} == "schedule" ]]; then
fuzz_type="fuzzing"
fi
echo "Github event: ${{ github.event_name }}"
echo "Fuzzing type: $fuzz_type"

for f in $(find . -name \*_fuzz.go); do
FUZZER_DIRECTORY=$(dirname $f)
echo "go-fuzz-build func ${fuzzers_funcs[$f]} residing in $f"
go-fuzz-build -func "${fuzzers_funcs[$f]}" -libfuzzer -o "$FUZZER_DIRECTORY/${fuzzers_targets[$f]}.a" $FUZZER_DIRECTORY
echo "Generating fuzzer binary of func ${fuzzers_funcs[$f]} which resides in $f"
clang-7 -fsanitize=fuzzer "$FUZZER_DIRECTORY/${fuzzers_targets[$f]}.a" -o "$FUZZER_DIRECTORY/${fuzzers_targets[$f]}"
fuzzit create job caddyserver/${fuzzers_targets[$f]} $FUZZER_DIRECTORY/${fuzzers_targets[$f]} --api-key ${FUZZIT_API_KEY} --type "${fuzz_type}" --branch "${SYSTEM_PULLREQUEST_SOURCEBRANCH}" --revision "${BUILD_SOURCEVERSION}"
echo "Completed $f"
done
Loading