Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust CI to handle build executables for supported platforms #176

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 7 additions & 33 deletions .github/actions/build/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,49 +14,23 @@ outputs:
runs:
using: "composite"
steps:
- if: runner.os == 'Linux'
name: Install apt packages
shell: bash
run: sudo apt-get install -y curl git git-lfs zip build-essential entr libssl-dev libdouble-conversion-dev pkg-config ninja-build cmake zlib1g-dev libffi-dev clang libclang-dev llvm llvm-dev libzip-dev libbz2-dev doctest-dev libboost-all-dev gcc g++ libgc-dev
- if: runner.os == 'macOS'
name: Install brew packages
shell: bash
run: HOMEBREW_NO_AUTO_UPDATE=1 brew install curl git git-lfs zip entr openssl double-conversion pkg-config ninja python cmake gnupg zlib doctest boost libzip lbzip2 llvm@19
- name: Restore cached Clang/LLVM
id: restore-clang-cache
uses: actions/cache/restore@v4
with:
path: |
${{ github.workspace }}/compiler+runtime/build
key: ${{ runner.os }}-${{ hashFiles('compiler+runtime/bin/build-clang') }}
- name: Compile and test
shell: bash
working-directory: ${{ github.workspace }}/compiler+runtime
run: ${{ github.workspace }}/compiler+runtime/bin/ci/test -DCMAKE_BUILD_TYPE=${{ matrix.build_type }}
- name: Store cached Clang/LLVM
id: save-clang-cache
uses: actions/cache/save@v4
with:
path: |
${{ github.workspace }}/compiler+runtime/build
key: ${{ steps.restore-clang-cache.outputs.cache-primary-key }}
# We need to validate formatting after building, since we build Clang/LLVM as part of our
# build process and we need clang-format from that local Clang build in order to have
# consistent formatting results.
run: ./bin/ci/test

- name: Validate formatting
shell: bash
working-directory: ${{ github.workspace }}/compiler+runtime
run: ${{ github.workspace }}/compiler+runtime/bin/ci/validate-formatting
run: ./bin/ci/validate-formatting

- if: ${{ env.CODECOV }} == "on"
name: Upload report to Codecov
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4

- if: ${{ inputs.update_release }}
name: Release
id: release
shell: bash
working-directory: ${{ github.workspace }}/compiler+runtime
run: |
export DESTDIR=${{ github.workspace }}/jank-${{ matrix.os }}-$(date +'%Y-%m-%d').$(git rev-parse --short $GITHUB_SHA)
${{ github.workspace }}/compiler+runtime/bin/install
tar czf $DESTDIR.tar.gz $DESTDIR/
echo "archive=$DESTDIR.tar.gz" >> $GITHUB_OUTPUT
run: ./bin/ci/install
62 changes: 39 additions & 23 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,34 +5,51 @@ on:
push:

jobs:
test:
linux:
strategy:
matrix:
build_type: [ Debug, Release ]
runs-on: ubuntu-latest
container: ubuntu:24.10
env:
CC: clang-19
CXX: clang++-19
timeout-minutes: 240
steps:
- name: Install deps
run: apt update && apt install -y git git-lfs sudo build-essential libzip-dev libbz2-dev libgc-dev clang-19 libboost-all-dev ninja-build doctest-dev cmake libllvmlibc-19-dev libclang-19-dev libssl-dev
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Build and test
uses: ./.github/actions/build
id: build
with:
update_release: ${{ github.ref == 'refs/heads/main' && matrix.build_type == 'Release' }}
- name: Update latest release
if: success() && github.ref == 'refs/heads/main' && ${{ matrix.build_type == 'Release' }}
uses: softprops/action-gh-release@v2
with:
tag_name: latest
token: ${{ secrets.GITHUB_TOKEN }}
files: ${{ steps.build.outputs.release_archive }}

macos:
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04]
os: [ macos-13, macos-15 ]
# TODO: Enable release once we no longer have a Clang crash:
# https://github.com/jank-lang/jank/actions/runs/12269972895/job/34234432107
build_type: [Debug, Release]
sanitize: [none, address, undefined, thread]
exclude:
- build_type: Release
sanitize: address
- build_type: Release
sanitize: undefined
- build_type: Release
sanitize: thread
include:
- os: ubuntu-24.04
build_type: Debug
codecov: on
analysis: on
sanitize: none
runs-on: ${{ matrix.os }}
env:
CODECOV: ${{ matrix.codecov }}
ANALYSIS: ${{ matrix.analysis }}
SANITIZE: ${{ matrix.sanitize }}
ASAN_OPTIONS: detect_leaks=0
CC: clang
CXX: clang++
timeout-minutes: 240 # 4h
steps:
- name: Install dependencies
run: HOMEBREW_NO_AUTO_UPDATE=1 brew install ninja libzip doctest boost llvm@19 double-conversion bdw-gc clang-format
- uses: actions/checkout@v4
with:
submodules: recursive
Expand All @@ -43,9 +60,8 @@ jobs:
update_release: ${{ github.ref == 'refs/heads/main' && matrix.build_type == 'Release' }}
- name: Update latest release
if: success() && github.ref == 'refs/heads/main' && ${{ matrix.build_type == 'Release' }}
uses: pyTooling/Actions/releaser@main
uses: softprops/action-gh-release@v2
with:
tag: latest
rm: true
tag_name: latest
token: ${{ secrets.GITHUB_TOKEN }}
files: ${{ steps.build.outputs.release_archive }}
32 changes: 16 additions & 16 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
[submodule "third-party/vcpkg"]
path = compiler+runtime/third-party/vcpkg
url = https://github.com/jank-lang/vcpkg.git
path = compiler+runtime/third-party/vcpkg
url = https://github.com/jank-lang/vcpkg.git
[submodule "compiler+runtime/third-party/folly"]
Comment on lines 1 to 4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very odd to see changes in here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tabs to spaces. My editor converts them automatically on save.

path = compiler+runtime/third-party/folly
url = https://github.com/jank-lang/folly.git
path = compiler+runtime/third-party/folly
url = https://github.com/jank-lang/folly.git
[submodule "compiler+runtime/third-party/bpptree"]
path = compiler+runtime/third-party/bpptree
url = https://github.com/jank-lang/BppTree.git
path = compiler+runtime/third-party/bpptree
url = https://github.com/jank-lang/BppTree.git
[submodule "compiler+runtime/third-party/fmt"]
path = compiler+runtime/third-party/fmt
url = https://github.com/jank-lang/fmt.git
path = compiler+runtime/third-party/fmt
url = https://github.com/jank-lang/fmt.git
[submodule "compiler+runtime/third-party/immer"]
path = compiler+runtime/third-party/immer
url = https://github.com/jank-lang/immer.git
path = compiler+runtime/third-party/immer
url = https://github.com/jank-lang/immer.git
[submodule "compiler+runtime/third-party/cli11"]
path = compiler+runtime/third-party/cli11
url = https://github.com/jank-lang/CLI11.git
path = compiler+runtime/third-party/cli11
url = https://github.com/jank-lang/CLI11.git
[submodule "compiler+runtime/third-party/libzippp"]
path = compiler+runtime/third-party/libzippp
url = https://github.com/jank-lang/libzippp.git
path = compiler+runtime/third-party/libzippp
url = https://github.com/jank-lang/libzippp.git
[submodule "compiler+runtime/third-party/boost-preprocessor"]
path = compiler+runtime/third-party/boost-preprocessor
url = https://github.com/jank-lang/boost-preprocessor.git
path = compiler+runtime/third-party/boost-preprocessor
url = https://github.com/jank-lang/boost-preprocessor.git
2 changes: 2 additions & 0 deletions compiler+runtime/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,8 @@ include(cmake/dependency/fmt.cmake)
include(cmake/dependency/libzippp.cmake)

find_package(OpenSSL REQUIRED COMPONENTS Crypto)
set(Boost_USE_STATIC_LIBS ON)
set(Boost_USE_STATIC_RUNTIME OFF)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the static runtime? Will you please put it as a comment so I can also know in 6 months when I ask myself the same thing? 😂

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can actually remove that, thanks for spotting

find_package(Boost CONFIG REQUIRED COMPONENTS filesystem)
find_package(BDWgc REQUIRED)

Expand Down
13 changes: 13 additions & 0 deletions compiler+runtime/bin/ci/install
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env bash

set -euo pipefail

export OS_NAME=$(uname | tr "[:upper:]" "[:lower:]")
export FOLDER=jank-${OS_NAME}-$(uname -m)
export DESTDIR=$(pwd)/${FOLDER}
./bin/install

cp -R ${DESTDIR} .

tar czf ${DESTDIR}.tar.gz ${FOLDER}/
echo "archive=${DESTDIR}.tar.gz" >> $GITHUB_OUTPUT
34 changes: 22 additions & 12 deletions compiler+runtime/bin/ci/test
Original file line number Diff line number Diff line change
@@ -1,20 +1,30 @@
#!/usr/bin/env bash

set -eu
set -e
if [[ "$OSTYPE" == "darwin"* ]];
then
export SDKROOT=$(xcrun --sdk macosx --show-sdk-path)
export PATH="$(brew --prefix)/opt/llvm/bin:${PATH}"
export LDFLAGS="-Wl,-rpath,$(brew --prefix)/opt/llvm/lib ${LDFLAGS}"
export CPPFLAGS="-I$(brew --prefix)/opt/llvm/include ${CPPFLAGS}"
fi

llvm_dir="${PWD}/build/llvm-install"
if [[ ! -d "${llvm_dir}" ]];
if [[ "${BUILD_LLVM:-off}" == "on" ]];
then
mkdir -p build
pushd build
../bin/build-clang
popd
llvm_dir="${PWD}/build/llvm-install"
if [[ ! -d "${llvm_dir}" ]];
then
mkdir -p build
pushd build
../bin/build-clang
popd
fi
export CC="${llvm_dir}/usr/local/bin/clang"
export CXX="${llvm_dir}/usr/local/bin/clang++"
fi
export CC="${llvm_dir}/usr/local/bin/clang"
export CXX="${llvm_dir}/usr/local/bin/clang++"

./bin/configure -GNinja \
-Djank_local_clang=on \
-Djank_local_clang="${BUILD_LLVM:-off}" \
-Djank_tests=on \
-Djank_analysis="${ANALYSIS:-off}" \
-Djank_sanitize="${SANITIZE:-none}" \
Expand All @@ -30,8 +40,8 @@ then
-Djank_analysis=off \
"$@"
LLVM_PROFILE_FILE=build/test.profraw ./bin/test
"${llvm_dir}/usr/local/bin/llvm-profdata" merge --sparse build/test.profraw -o build/test.profdata
"${llvm_dir}/usr/local/bin/llvm-cov" show ./build/jank-test --instr-profile build/test.profdata > coverage.txt
"${llvm_dir:-/}/usr/local/bin/llvm-profdata" merge --sparse build/test.profraw -o build/test.profdata
"${llvm_dir:-/}/usr/local/bin/llvm-cov" show ./build/jank-test --instr-profile build/test.profdata > coverage.txt
# This file will be read by a subsequent CodeCov Github action.
else
./bin/test
Expand Down
6 changes: 5 additions & 1 deletion compiler+runtime/bin/ci/validate-formatting
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@
set -euo pipefail

success=true
clang_format="./build/llvm-install/usr/local/bin/clang-format"
if [[ -e "./build/llvm-install" ]]; then
clang_format="./build/llvm-install/usr/local/bin/clang-format"
else
clang_format="$(command -v clang-format)"
fi
Comment on lines +6 to +10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please drop the then down, Allman style. Also, this file has 2 space indentation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth adding an editorconfig file, I can add also if you want

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you if it's worth it.


while IFS= read -r -d '' file
do
Expand Down
2 changes: 1 addition & 1 deletion compiler+runtime/test/cpp/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
int main(int const argc, char const **argv)
try
{
std::locale::global(std::locale(""));
std::locale::global(std::locale("en_US.UTF-8"));

GC_set_all_interior_pointers(1);
GC_enable();
Expand Down