Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Next-generation CI/CD pipelines with RunsOn #11001

Draft
wants to merge 48 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
1fa8c70
Remove existing GH workflows
hcho3 Oct 17, 2024
78c4d8c
First RunsOn example
hcho3 Oct 17, 2024
92fd6a2
Test custom runner config
hcho3 Oct 17, 2024
d6761cb
Try out GPU support
hcho3 Oct 18, 2024
3331cd6
Add Windows
hcho3 Oct 22, 2024
61f207d
Fix Windows build
hcho3 Oct 23, 2024
6b78a12
Custom image for Linux
hcho3 Oct 23, 2024
000be18
New Docker wrapper with S3 caching
hcho3 Oct 24, 2024
e1e3b41
Create utility script to build CI container
hcho3 Oct 24, 2024
5373276
Migrate some tasks from BuildKite
hcho3 Oct 25, 2024
fa96af1
Move more pipelines; refine stash_artifacts
hcho3 Oct 29, 2024
f387555
Simplify stash_artifacts
hcho3 Oct 30, 2024
91eee2d
Migrate Python tests
hcho3 Oct 30, 2024
26fff38
Mass renaming; Migrate R GPU pkg build + MacOS
hcho3 Oct 30, 2024
bf4b8ea
Fix dmlc/xgboost#10752
hcho3 Nov 1, 2024
80a883e
Fix clang-tidy + rpkg build
hcho3 Nov 1, 2024
d86deda
[WIP] Properly handle shaded JARs
hcho3 Nov 5, 2024
125b7e9
[CI] Pin Dask to 2024.10.0 (dmlc/xgboost#10995)
hcho3 Nov 13, 2024
f3ccc6f
Refactor JVM tests
hcho3 Nov 13, 2024
67d0cc6
[jvm-packages] resolve spark compatibility issue (#10917)
wbo4958 Oct 23, 2024
b65e2cc
Test GPU JVM packages
hcho3 Nov 14, 2024
f4d94a1
Disable the host numa virtual memory allocator for now. (#10934)
trivialfis Nov 2, 2024
e1c7e24
Run GPU tests with privilege escalation
hcho3 Nov 15, 2024
11b0427
Reboot after install driver; use proprietary driver for now
hcho3 Nov 17, 2024
49eaec1
Try removing --privileged flag
hcho3 Nov 17, 2024
d3482e1
Revert "Disable the host numa virtual memory allocator for now. (#109…
hcho3 Nov 17, 2024
130d303
Build JVM docs
hcho3 Nov 18, 2024
a45b24f
Re-org test scripts
hcho3 Nov 18, 2024
6dc0df2
More reorg
hcho3 Nov 18, 2024
d1d82bc
Disable the host numa virtual memory allocator for now. (#10934)
trivialfis Nov 2, 2024
ad75358
Fix run flags to Python tests
hcho3 Nov 18, 2024
06d060b
Separate workflow for JVM packages
hcho3 Nov 19, 2024
588dd67
Rename workflow files
hcho3 Nov 19, 2024
30c1470
Merge remote-tracking branch 'upstream/master'
hcho3 Nov 19, 2024
cb8f63f
Consolidate workflow defs
hcho3 Nov 19, 2024
9f9db2b
Various fixes
hcho3 Nov 19, 2024
dce4f7b
Disable dependabot for now
hcho3 Nov 19, 2024
dfd5624
Fixes
hcho3 Nov 19, 2024
0ee55c2
Don't upload artifacts from pull requests
hcho3 Nov 19, 2024
90594ff
Merge branch 'master' into xgboost-ci-ng
hcho3 Nov 19, 2024
cb55d7a
Fix
hcho3 Nov 19, 2024
377b3e7
Merge remote-tracking branch 'upstream/master'
hcho3 Nov 19, 2024
6641f7d
Fix merge conflict
hcho3 Nov 19, 2024
0727cf5
Fix
hcho3 Nov 19, 2024
3c2e5c8
Remove unneeded matrix; set max-parallel
hcho3 Nov 19, 2024
32f7406
Formatting fixes
hcho3 Nov 20, 2024
e148e29
Fix update_rapids.sh
hcho3 Nov 20, 2024
d8abb3c
Add JVM deploy
hcho3 Nov 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 0 additions & 35 deletions .github/dependabot.yml

This file was deleted.

1 change: 0 additions & 1 deletion .github/runs-on.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,3 @@ runners:
cpu: 32
family: ["c7i-flex", "c7i", "c7a", "c5", "c5a"]
image: windows-amd64

32 changes: 16 additions & 16 deletions .github/workflows/freebsd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,20 @@ jobs:
timeout-minutes: 20
name: A job to run test in FreeBSD
steps:
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: Test in FreeBSD
id: test
uses: vmactions/freebsd-vm@v1
with:
usesh: true
prepare: |
pkg install -y cmake git ninja googletest
- uses: actions/checkout@v4
with:
submodules: 'true'
- name: Test in FreeBSD
id: test
uses: vmactions/freebsd-vm@v1
with:
usesh: true
prepare: |
pkg install -y cmake git ninja googletest

run: |
mkdir build
cd build
cmake .. -GNinja -DGOOGLE_TEST=ON
ninja -v
./testxgboost
run: |
mkdir build
cd build
cmake .. -GNinja -DGOOGLE_TEST=ON
ninja -v
./testxgboost
44 changes: 22 additions & 22 deletions .github/workflows/i386.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,25 @@ jobs:
ports:
- 5000:5000
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
submodules: 'true'
- name: Set up Docker Buildx
uses: docker/[email protected]
with:
driver-opts: network=host
- name: Build and push container
uses: docker/build-push-action@v6
with:
context: .
file: tests/ci_build/Dockerfile.i386
push: true
tags: localhost:5000/xgboost/build-32bit:latest
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build XGBoost
run: |
docker run --rm -v $PWD:/workspace -w /workspace \
-e CXXFLAGS='-Wno-error=overloaded-virtual -Wno-error=maybe-uninitialized -Wno-error=redundant-move' \
localhost:5000/xgboost/build-32bit:latest \
tests/ci_build/build_via_cmake.sh
- uses: actions/checkout@v4.2.2
with:
submodules: 'true'
- name: Set up Docker Buildx
uses: docker/[email protected]
with:
driver-opts: network=host
- name: Build and push container
uses: docker/build-push-action@v6
with:
context: .
file: ops/docker/dockerfile/Dockerfile.i386
push: true
tags: localhost:5000/xgboost/build-32bit:latest
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build XGBoost
run: |
docker run --rm -v $PWD:/workspace -w /workspace \
-e CXXFLAGS='-Wno-error=overloaded-virtual -Wno-error=maybe-uninitialized -Wno-error=redundant-move' \
localhost:5000/xgboost/build-32bit:latest \
bash ops/script/build_via_cmake.sh
Loading
Loading