Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend CI for Ubuntu 22.04 #686

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/scripts/ubuntu-20.04/compile_build.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

# Compile dependencies that cannot be acquired via the official repositories

mkdir -p ${INSTALL_PATH}
Expand Down
9 changes: 8 additions & 1 deletion .github/scripts/ubuntu-20.04/download_build.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#!/bin/bash

# Download installers for compiled dependencies

mkdir -p ${DOWNLOAD_PATH}
Expand All @@ -20,4 +22,9 @@ hash_value=$(echo -n "$concatenated_hashes" | md5sum | awk '{print $1}')

## Save said hash

echo "compile-hash-key=${hash_value}" >> $GITHUB_OUTPUT

### Restore action
echo "ubuntu-20-04-compile-hash-key=${hash_value}" >> "$GITHUB_OUTPUT"

### Save action
echo "CACHE_KEY=${hash_value}" >> "$GITHUB_ENV"
5 changes: 5 additions & 0 deletions .github/scripts/ubuntu-20.04/postcompile_runtime.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash

# Post compilation configuration for runtime (environmental variables, library location settings etc..)

echo "LD_LIBRARY_PATH=${INSTALL_PATH}/thrift/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
30 changes: 30 additions & 0 deletions .github/scripts/ubuntu-22.04/compile_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/bin/bash

# Compile dependencies that cannot be acquired via the official repositories

mkdir -p ${INSTALL_PATH}

## Compile build2

sh "${DOWNLOAD_PATH}/install_latest_build2.sh" ${INSTALL_PATH}/build2

## Compile odb, libodb, and its connectors

mkdir -p ${DOWNLOAD_PATH}/odb
cd ${DOWNLOAD_PATH}/odb
${INSTALL_PATH}/build2/bin/bpkg create --quiet --jobs $(nproc) cc \
config.cxx=g++ \
config.cc.coptions=-O3 \
config.bin.rpath=${INSTALL_PATH}/odb/lib \
config.install.root=${INSTALL_PATH}/odb

### Getting the source
${INSTALL_PATH}/build2/bin/bpkg add https://pkg.cppget.org/1/beta --trust-yes
${INSTALL_PATH}/build2/bin/bpkg fetch --trust-yes

### Building odb
${INSTALL_PATH}/build2/bin/bpkg build odb --yes
${INSTALL_PATH}/build2/bin/bpkg build libodb --yes
${INSTALL_PATH}/build2/bin/bpkg build libodb-sqlite --yes
${INSTALL_PATH}/build2/bin/bpkg build libodb-pgsql --yes
${INSTALL_PATH}/build2/bin/bpkg install --all --recursive
23 changes: 23 additions & 0 deletions .github/scripts/ubuntu-22.04/download_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash

# Download installers for compiled dependencies

mkdir -p "${DOWNLOAD_PATH}"

## Thrift 0.16 + ODB beta

wget -O "${DOWNLOAD_PATH}/install_latest_build2.sh" "https://github.com/Ericsson/CodeCompass/raw/master/scripts/install_latest_build2.sh"
build2_version=$(sh "${DOWNLOAD_PATH}/install_latest_build2.sh" --version)
odb_signature=$(wget -qO- https://pkg.cppget.org/1/beta/signature.manifest)

# Calculate hash of dependencies for Github Cache Action

hash_value=$(echo -n "${build2_version}${odb_signature}" | md5sum | awk '{print $1}')

## Save said hash

### Restore action
echo "ubuntu-22-04-compile-hash-key=${hash_value}" >> "$GITHUB_OUTPUT"

### Save action
echo "CACHE_KEY=${hash_value}" >> "$GITHUB_ENV"
6 changes: 6 additions & 0 deletions .github/scripts/ubuntu-22.04/postcompile_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/bash

# Post compilation configuration for building (environmental variables, library location settings etc..)

echo "${INSTALL_PATH}/odb/bin" >> $GITHUB_PATH
echo "CMAKE_PREFIX_PATH=${INSTALL_PATH}/odb:$CMAKE_PREFIX_PATH" >> $GITHUB_ENV
5 changes: 5 additions & 0 deletions .github/scripts/ubuntu-22.04/postcompile_runtime.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash

# Post compilation configuration for runtime (environmental variables, library location settings etc..)

echo "LD_LIBRARY_PATH=${INSTALL_PATH}/odb/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
8 changes: 8 additions & 0 deletions .github/scripts/ubuntu-22.04/setup_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash

# Install required packages for CodeCompass build
sudo apt install git cmake make g++ libboost-all-dev \
llvm-11-dev clang-11 libclang-11-dev \
gcc-11-plugin-dev thrift-compiler libthrift-dev \
default-jdk libssl-dev libgraphviz-dev libmagic-dev libgit2-dev exuberant-ctags doxygen \
libldap2-dev libgtest-dev
4 changes: 4 additions & 0 deletions .github/scripts/ubuntu-22.04/setup_postgresql.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash

# Install PostgreSQL
sudo apt-get install postgresql-server-dev-14
4 changes: 4 additions & 0 deletions .github/scripts/ubuntu-22.04/setup_sqlite3.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash

# Install SQLite3
sudo apt-get install libsqlite3-dev
47 changes: 28 additions & 19 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,18 @@ env:
permissions: read-all

jobs:



## BUILD JOBS
build:
strategy:
matrix:
db: [postgresql, sqlite3]
os: [ubuntu-20.04]
os: [ubuntu-20.04, ubuntu-22.04]
fail-fast: false

runs-on: ${{ matrix.os }}
outputs:
has-compiled-dependencies: ${{ steps.compilation-flag.outputs.has-compiled-dependencies }}
compile-hash-key: ${{ steps.download-compile-dependencies.outputs.compile-hash-key }}
ubuntu-20-04-compile-hash-key: ${{ steps.download-compile-dependencies.outputs.compile-hash-key }}
ubuntu-22-04-compile-hash-key: ${{ steps.download-compile-dependencies.outputs.compile-hash-key }}

services:
# Label used to access the service container
Expand Down Expand Up @@ -58,47 +55,46 @@ jobs:
id: compilation-flag
run: |
if [ -f ./.github/scripts/${{ matrix.os }}/compile_build.sh ]; then
echo "has-compiled-dependencies=true" >> "$GITHUB_OUTPUT"
echo "HAS_COMPILED_DEPENDENCIES=true" >> "$GITHUB_ENV"
else
echo "has-compiled-dependencies=false" >> "$GITHUB_OUTPUT"
echo "HAS_COMPILED_DEPENDENCIES=false" >> "$GITHUB_ENV"
fi

- name: Download installers for compiled dependencies
if: steps.compilation-flag.outputs.has-compiled-dependencies == 'true'
if: ${{ env.HAS_COMPILED_DEPENDENCIES == 'true' }}
id: download-compile-dependencies
run: |
chmod +x ./.github/scripts/${{ matrix.os }}/download_build.sh
./.github/scripts/${{ matrix.os }}/download_build.sh

- name: Restore compiled dependencies
if: steps.compilation-flag.outputs.has-compiled-dependencies == 'true'
id: restore-compiled-dependencies
uses: actions/cache/restore@v3
with:
path: ${{ env.INSTALL_PATH }}
key: ${{ matrix.os }}-compile-install-${{steps.download-compile-dependencies.outputs.compile-hash-key}}
key: ${{ matrix.os }}-compile-install-${{ env.CACHE_KEY }}

- name: Compile dependencies
if: steps.restore-compiled-dependencies.outputs.cache-hit != 'true' && steps.compilation-flag.outputs.has-compiled-dependencies == 'true'
if: ${{ env.HAS_COMPILED_DEPENDENCIES == 'true' && steps.restore-compiled-dependencies.outputs.cache-hit != 'true' }}
run: |
chmod +x ./.github/scripts/${{ matrix.os }}/compile_build.sh
./.github/scripts/${{ matrix.os }}/compile_build.sh

- name: Post compilation configuration (build)
if: steps.compilation-flag.outputs.has-compiled-dependencies == 'true'
if: ${{ env.HAS_COMPILED_DEPENDENCIES == 'true' }}
run: |
if [ -f ./.github/scripts/${{ matrix.os }}/postcompile_build.sh ]; then
chmod +x ./.github/scripts/${{ matrix.os }}/postcompile_build.sh
./.github/scripts/${{ matrix.os }}/postcompile_build.sh
fi

- name: Save dependencies
if: steps.restore-compiled-dependencies.outputs.cache-hit != 'true' && steps.compilation-flag.outputs.has-compiled-dependencies == 'true'
if: ${{ env.HAS_COMPILED_DEPENDENCIES == 'true' && steps.restore-compiled-dependencies.outputs.cache-hit != 'true' }}
id: save-compiled-dependencies
uses: actions/cache/save@v3
with:
path: ${{ env.INSTALL_PATH }}
key: ${{ matrix.os }}-compile-install-${{steps.download-compile-dependencies.outputs.compile-hash-key}}
key: ${{ matrix.os }}-compile-install-${{ env.CACHE_KEY }}

- name: Install database packages
run: ./.github/scripts/${{ matrix.os }}/setup_${{matrix.db}}.sh
Expand Down Expand Up @@ -186,7 +182,7 @@ jobs:
strategy:
matrix:
db: [postgresql, sqlite3]
os: [ubuntu-20.04]
os: [ubuntu-20.04, ubuntu-22.04]
fail-fast: false

runs-on: ${{ matrix.os }}
Expand Down Expand Up @@ -222,16 +218,29 @@ jobs:
- name: Install database packages
run: ./.github/scripts/${{ matrix.os }}/setup_${{matrix.db}}.sh

- name: Set has-compiled-dependencies flag
id: compilation-flag
run: |
if [ -f ./.github/scripts/${{ matrix.os }}/compile_build.sh ]; then
echo "HAS_COMPILED_DEPENDENCIES=true" >> "$GITHUB_ENV"
else
echo "HAS_COMPILED_DEPENDENCIES=false" >> "$GITHUB_ENV"
fi

- name: Restore compiled dependencies
if: needs.build.outputs.has-compiled-dependencies == 'true'
if: ${{ env.HAS_COMPILED_DEPENDENCIES == 'true' }}
id: restore-compiled-dependencies
uses: actions/cache/restore@v3
with:
path: ${{ env.INSTALL_PATH }}
key: ${{ matrix.os }}-compile-install-${{needs.build.outputs.compile-hash-key}}
fail-on-cache-miss: true
key: ${{ matrix.os }}-compile-install-${{ needs.build.outputs.ubuntu-20-04-compile-hash-key }}
restore-keys: |
${{ matrix.os }}-compile-install-${{ needs.build.outputs.ubuntu-20-04-compile-hash-key }}
${{ matrix.os }}-compile-install-${{ needs.build.outputs.ubuntu-22-04-compile-hash-key }}

- name: Post compilation configuration (runtime)
if: needs.build.outputs.has-compiled-dependencies == 'true'
if: ${{ env.HAS_COMPILED_DEPENDENCIES == 'true' }}
run: |
if [ -f ./.github/scripts/${{ matrix.os }}/postcompile_runtime.sh ]; then
chmod +x ./.github/scripts/${{ matrix.os }}/postcompile_runtime.sh
Expand Down
15 changes: 14 additions & 1 deletion scripts/install_latest_build2.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,23 @@ fi

install_dir=$1
toolchain_file="toolchain.sha256"
wget --no-verbose --no-clobber https://download.build2.org/toolchain.sha256 -O "${toolchain_file}"
if [ "$1" = "--version" ]; then
wget -q --no-verbose --no-clobber https://download.build2.org/toolchain.sha256 -O "${toolchain_file}"
else
wget --no-verbose --no-clobber https://download.build2.org/toolchain.sha256 -O "${toolchain_file}"
fi
mcserep marked this conversation as resolved.
Show resolved Hide resolved


version_line=$(grep -m 1 '' "$toolchain_file")
version_number=$(echo "$version_line" | awk '{print $2}')

### Return with version string only without actually installing build2 if "--version" flag is specified.
if [ "$1" = "--version" ]; then
echo "${version_number}"
rm -f "${toolchain_file}"
exit 0
fi

version_to_install=build2-install-${version_number}.sh
download_url=https://download.build2.org/${version_number}/${version_to_install}
wget --no-verbose --no-clobber "${download_url}" -O "${version_to_install}"
Expand Down
Loading