-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update build system for Linux, macOS, overhaul GitHub Actions, remove serial driver #37
Open
garrettsummerfi3ld
wants to merge
9
commits into
REVrobotics:main
Choose a base branch
from
unofficial-rev-port:fix/upstream-build-changes
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
8c9f640
Update GitHub Actions workflows, update GoogleTest dependency
garrettsummerfi3ld 06b9a96
Add IOKit linker args for macOS, disable errors as warnings for all p…
garrettsummerfi3ld 1d90703
Remove Windows-only preprocessor macros
garrettsummerfi3ld 895fea5
Add a versioning check for CI
garrettsummerfi3ld 744b1dd
Remove serial driver
garrettsummerfi3ld c446000
Create Dependabot update task
garrettsummerfi3ld a292011
Fix CI: Make curl follow redirects
ItsHarper 4e47f70
Include both static and shared WPILib artifacts
ItsHarper 2fa667d
Remove code duplication in build workflow
qwertychouskie File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
version: 2 | ||
updates: | ||
# Dependabot updates for GitHub Actions | ||
- package-ecosystem: github-actions | ||
directory: "/" | ||
schedule: | ||
interval: "weekly" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,96 +1,203 @@ | ||
name: Build | ||
|
||
on: | ||
push: | ||
branches: | ||
- 'main' | ||
pull_request: | ||
branches: | ||
- '*' | ||
[push, pull_request] | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }} | ||
cancel-in-progress: false | ||
|
||
defaults: | ||
run: | ||
shell: bash | ||
|
||
jobs: | ||
# Use native runners to build for Windows, Linux x86-64, and macOS; use WPI provided Docker images to build for Linux ARM32 and ARM64 platforms | ||
build: | ||
timeout-minutes: 15 | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
include: | ||
- os: windows-latest | ||
container: '' | ||
name: windows64 | ||
name: "build-${{ matrix.name }}" | ||
name: Win64 | ||
build-options: "" | ||
platform-type: windowsx86-64 | ||
- os: ubuntu-latest | ||
name: Linux64 | ||
platform-type: linuxx86-64 | ||
build-options: "" | ||
- os: macos-latest | ||
name: macOS | ||
platform-type: osxuniversal | ||
build-options: "" | ||
- container: wpilib/aarch64-cross-ubuntu:bullseye-22.04 | ||
os: ubuntu-latest | ||
name: LinuxARM64 | ||
build-options: "-Ponlylinuxarm64" | ||
platform-type: linuxarm64 | ||
arch: arm64 | ||
- container: wpilib/raspbian-cross-ubuntu:bullseye-22.04 | ||
os: ubuntu-latest | ||
name: LinuxARM32 | ||
build-options: "-Ponlylinuxarm32" | ||
platform-type: linuxarm32 | ||
arch: arm32 | ||
name: "Build - ${{ matrix.name }}" | ||
runs-on: ${{ matrix.os }} | ||
container: ${{ matrix.container }} | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v3 | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.sha }} | ||
|
||
- name: Setup Java | ||
uses: actions/setup-java@v3 | ||
uses: actions/setup-java@v4 | ||
with: | ||
distribution: 'zulu' | ||
java-version: 11 | ||
|
||
- name: Setup Gradle | ||
uses: gradle/actions/setup-gradle@v4 | ||
|
||
- name: Build | ||
run: | | ||
./gradlew outputVersions publish ${{ matrix.build-options }} -PreleaseMode | ||
|
||
- name: Download WPILib HAL artifacts and headers, gather all needed headers | ||
- name: Download WPILib HAL artifacts and headers for ${{ matrix.platform-type }} | ||
run : | | ||
halVersion=$(cat wpiHalVersion.txt) | ||
|
||
sharedHalPlatformUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/hal/hal-cpp/"$halVersion"/hal-cpp-"$halVersion"-${{ matrix.platform-type }}.zip | ||
sharedUtilPlatformUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/wpiutil/wpiutil-cpp/"$halVersion"/wpiutil-cpp-"$halVersion"-${{ matrix.platform-type }}.zip | ||
curl -L -o sharedHalPlatform.zip "$sharedHalPlatformUrl" | ||
curl -L -o sharedUtilPlatform.zip "$sharedUtilPlatformUrl" | ||
|
||
# Download WPILib artifacts from Artifactory | ||
halWindowsUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/hal/hal-cpp/"$halVersion"/hal-cpp-"$halVersion"-windowsx86-64.zip | ||
staticHalPlatformUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/hal/hal-cpp/"$halVersion"/hal-cpp-"$halVersion"-${{ matrix.platform-type }}static.zip | ||
staticUtilPlatformUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/wpiutil/wpiutil-cpp/"$halVersion"/wpiutil-cpp-"$halVersion"-${{ matrix.platform-type }}static.zip | ||
curl -L -o staticHalPlatform.zip "$staticHalPlatformUrl" | ||
curl -L -o staticUtilPlatform.zip "$staticUtilPlatformUrl" | ||
|
||
- name: Unzip WPILib HAL artifacts and headers | ||
run: | | ||
unzip sharedHalPlatform.zip -d sharedHalPlatform | ||
unzip sharedUtilPlatform.zip -d sharedUtilPlatform | ||
unzip staticHalPlatform.zip -d staticHalPlatform | ||
unzip staticUtilPlatform.zip -d staticUtilPlatform | ||
mkdir -p CANBridge-artifacts/static | ||
mkdir -p CANBridge-artifacts/shared | ||
|
||
# Put release files together in one directory based on platform | ||
- name: Create Artifact | ||
run: | | ||
mkdir -p CANBridge-artifacts | ||
if [[ "${{ matrix.platform-type }}" == "windowsx86-64" ]]; then | ||
cp build/libs/cANBridge/shared/windowsx86-64/release/CANBridge.dll CANBridge-artifacts/shared/ | ||
cp build/libs/cANBridge/shared/windowsx86-64/release/CANBridge.lib CANBridge-artifacts/shared/ | ||
cp sharedHalPlatform/windows/x86-64/shared/wpiHal.dll CANBridge-artifacts/shared/ | ||
cp sharedHalPlatform/windows/x86-64/shared/wpiHal.lib CANBridge-artifacts/shared/ | ||
cp sharedUtilPlatform/windows/x86-64/shared/wpiutil.dll CANBridge-artifacts/shared/ | ||
cp sharedUtilPlatform/windows/x86-64/shared/wpiutil.lib CANBridge-artifacts/shared/ | ||
|
||
cp build/libs/cANBridge/static/windowsx86-64/release/CANBridge.lib CANBridge-artifacts/static/ | ||
cp staticHalPlatform/windows/x86-64/static/wpiHal.lib CANBridge-artifacts/static/ | ||
cp staticUtilPlatform/windows/x86-64/static/wpiutil.lib CANBridge-artifacts/static/ | ||
elif [[ "${{ matrix.platform-type }}" == "linuxx86-64" ]]; then | ||
cp build/libs/cANBridge/shared/linuxx86-64/release/libCANBridge.so CANBridge-artifacts/shared/ | ||
cp sharedHalPlatform/linux/x86-64/shared/libwpiHal.so CANBridge-artifacts/shared/ | ||
cp sharedUtilPlatform/linux/x86-64/shared/libwpiutil.so CANBridge-artifacts/shared/ | ||
|
||
cp build/libs/cANBridge/static/linuxx86-64/release/libCANBridge.a CANBridge-artifacts/static/ | ||
cp staticHalPlatform/linux/x86-64/static/libwpiHal.a CANBridge-artifacts/static/ | ||
cp staticUtilPlatform/linux/x86-64/static/libwpiutil.a CANBridge-artifacts/static/ | ||
elif [[ "${{ matrix.platform-type }}" == "osxuniversal" ]]; then | ||
cp build/libs/cANBridge/shared/osxuniversal/release/libCANBridge.dylib CANBridge-artifacts/shared/ | ||
cp sharedHalPlatform/osx/universal/shared/libwpiHal.dylib CANBridge-artifacts/shared/ | ||
cp sharedUtilPlatform/osx/universal/shared/libwpiutil.dylib CANBridge-artifacts/shared | ||
|
||
cp build/libs/cANBridge/static/osxuniversal/release/libCANBridge.a CANBridge-artifacts/static/ | ||
cp staticHalPlatform/osx/universal/static/libwpiHal.a CANBridge-artifacts/static/ | ||
cp staticUtilPlatform/osx/universal/static/libwpiutil.a CANBridge-artifacts/static/ | ||
elif [[ "${{ matrix.platform-type }}" == "linuxarm32" || "${{ matrix.platform-type }}" == "linuxarm64" ]]; then | ||
cp build/libs/cANBridge/shared/release/libCANBridge.so CANBridge-artifacts/shared/libCANBridge.so | ||
cp sharedHalPlatform/linux/${{ matrix.arch }}/shared/libwpiHal.so CANBridge-artifacts/shared/libwpiHal.so | ||
cp sharedUtilPlatform/linux/${{ matrix.arch }}/shared/libwpiutil.so CANBridge-artifacts/shared/libwpiutil.so | ||
|
||
cp build/libs/cANBridge/static/release/libCANBridge.a CANBridge-artifacts/static/libCANBridge.a | ||
cp staticHalPlatform/linux/${{ matrix.arch }}/static/libwpiHal.a CANBridge-artifacts/static/libwpiHal.a | ||
cp staticUtilPlatform/linux/${{ matrix.arch }}/static/libwpiutil.a CANBridge-artifacts/static/libwpiutil.a | ||
fi | ||
|
||
# Upload build artifact | ||
- name: Upload build artifact | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: CANBridge-${{ matrix.platform-type }}-${{ github.sha}} | ||
path: CANBridge-artifacts/ | ||
|
||
# Upload combined headers for WPILib from HAL and WPIUtil | ||
wpi-headers: | ||
runs-on: ubuntu-latest | ||
name: "WPILib Headers" | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.sha }} | ||
- name: Download WPILib HAL artifacts and headers | ||
run : | | ||
halVersion=$(cat wpiHalVersion.txt) | ||
|
||
halHeadersUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/hal/hal-cpp/"$halVersion"/hal-cpp-"$halVersion"-headers.zip | ||
utilWindowsUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/wpiutil/wpiutil-cpp/"$halVersion"/wpiutil-cpp-"$halVersion"-windowsx86-64.zip | ||
utilHeadersUrl=https://frcmaven.wpi.edu/artifactory/release/edu/wpi/first/wpiutil/wpiutil-cpp/"$halVersion"/wpiutil-cpp-"$halVersion"-headers.zip | ||
curl -L -o halWindows.zip "$halWindowsUrl" | ||
|
||
curl -L -o halHeaders.zip "$halHeadersUrl" | ||
curl -L -o utilWindows.zip "$utilWindowsUrl" | ||
curl -L -o utilHeaders.zip "$utilHeadersUrl" | ||
unzip halWindows.zip -d halWindows | ||
|
||
- name: Unzip WPILib HAL artifacts and headers | ||
run: | | ||
unzip halHeaders.zip -d halHeaders | ||
unzip utilWindows.zip -d utilWindows | ||
unzip utilHeaders.zip -d utilHeaders | ||
|
||
# Gather all of the the needed headers | ||
- name: Gather all needed headers | ||
run: | | ||
mkdir headers-for-artifact | ||
cp -r halHeaders/hal headers-for-artifact | ||
cp -r utilHeaders/wpi headers-for-artifact | ||
cp -r src/main/native/include/* headers-for-artifact | ||
|
||
# Zip the needed headers and put them in the appropriate location for artifact upload | ||
mkdir -p CANBridge-artifacts | ||
7z a CANBridge-artifacts/headers.zip ./headers-for-artifact/* | ||
|
||
# Put release files together in one directory | ||
- name: Create Artifact | ||
run: | | ||
mkdir -p CANBridge-artifacts | ||
cp build/libs/cANBridge/static/windowsx86-64/release/CANBridge.lib CANBridge-artifacts/CANBridge-static.lib | ||
cp build/libs/cANBridge/shared/windowsx86-64/release/CANBridge.dll CANBridge-artifacts/CANBridge.dll | ||
cp build/libs/cANBridge/shared/windowsx86-64/release/CANBridge.lib CANBridge-artifacts/CANBridge.lib | ||
cp halWindows/windows/x86-64/shared/wpiHal.dll CANBridge-artifacts/wpiHal.dll | ||
cp halWindows/windows/x86-64/shared/wpiHal.lib CANBridge-artifacts/wpiHal.lib | ||
cp utilWindows/windows/x86-64/shared/wpiutil.dll CANBridge-artifacts/wpiutil.dll | ||
cp utilWindows/windows/x86-64/shared/wpiutil.lib CANBridge-artifacts/wpiutil.lib | ||
|
||
# Upload build artifact | ||
- name: Upload build artifact | ||
uses: actions/upload-artifact@v3 | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: CANBridge-${{ github.sha }} | ||
path: CANBridge-artifacts/ | ||
path: headers-for-artifact | ||
name: headers | ||
|
||
# Upload version file, used for versioning | ||
version: | ||
runs-on: ubuntu-latest | ||
name: "Version" | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.sha }} | ||
|
||
- name: Setup Java | ||
uses: actions/setup-java@v4 | ||
with: | ||
distribution: 'zulu' | ||
java-version: 11 | ||
|
||
- name: Setup Gradle | ||
uses: gradle/actions/setup-gradle@v4 | ||
|
||
- name: Build | ||
run: | | ||
./gradlew outputVersions -PreleaseMode | ||
|
||
# Upload version.txt | ||
- name: Upload version artifact | ||
uses: actions/upload-artifact@v3 | ||
- name: Upload build artifact | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: version | ||
path: build/allOutputs/version.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,72 +3,86 @@ name: Create release | |
on: | ||
push: | ||
tags: | ||
- 'v*' | ||
- 'v**' | ||
|
||
permissions: | ||
contents: write | ||
|
||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }} | ||
cancel-in-progress: false | ||
|
||
defaults: | ||
run: | ||
shell: bash | ||
|
||
jobs: | ||
check-versions: | ||
# Checks previous build workflow and gets the version from publish.gradle | ||
check-build: | ||
name: Check build and publish versions | ||
runs-on: ubuntu-latest | ||
outputs: | ||
TAG_NAME: ${{ env.TAG_NAME }} | ||
VERSION: ${{ steps.get_version.outputs.version }} | ||
steps: | ||
- name: Wait for build to finish | ||
uses: lewagon/[email protected].1 | ||
- name: Wait for build workflow to finish | ||
uses: lewagon/[email protected].4 | ||
with: | ||
ref: ${{ github.ref }} | ||
check-name: 'build-windows64' | ||
check-regexp: 'Build|WPILib Headers|Version' | ||
repo-token: ${{ secrets.GITHUB_TOKEN }} | ||
wait-interval: 10 | ||
- name: Get tag name | ||
run: | | ||
echo "TAG_NAME=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV | ||
|
||
# Download artifacts from build workflow | ||
- name: Download workflow artifacts | ||
uses: dawidd6/action-download-artifact@v2 | ||
uses: dawidd6/action-download-artifact@v6 | ||
with: | ||
workflow: build.yml | ||
commit: ${{ github.sha }} | ||
path: '.' | ||
|
||
# Get publish.gradle version | ||
- name: Get publish.gradle version | ||
id: get_version | ||
id: get-version | ||
run: | | ||
echo "version=$(cat version/version.txt)" >> $GITHUB_OUTPUT | ||
echo "expectedTagName=v$(cat version/version.txt)" >> $GITHUB_OUTPUT | ||
|
||
# Check publish.gradle version | ||
- name: publish.gradle version check FAILED | ||
if: ${{ steps.get_version.outputs.expectedTagName != env.TAG_NAME }} | ||
# Check if the publish.gradle version matches the tag name | ||
- name: Check version | ||
run: | | ||
echo Tag name: ${{ env.TAG_NAME }} | ||
echo publish.gradle version: ${{ steps.get_version.outputs.version }} | ||
exit 1 | ||
if [[ "${{ steps.get-version.outputs.expectedTagName }}" != "${{ env.TAG_NAME }}" ]]; then | ||
echo "Version mismatch: ${{ steps.get-version.outputs.expectedTagName }} != ${{ env.TAG_NAME }}" | ||
exit 1 | ||
fi | ||
|
||
# Creates a release draft with the artifacts from the build workflow | ||
prepare-release: | ||
name: Prepare release | ||
runs-on: ubuntu-latest | ||
needs: check-versions | ||
needs: check-build | ||
steps: | ||
# Download API, docs, and version.txt | ||
- name: Download workflow artifacts | ||
uses: dawidd6/action-download-artifact@v2 | ||
uses: dawidd6/action-download-artifact@v6 | ||
with: | ||
workflow: build.yml | ||
commit: ${{ github.sha }} | ||
path: '.' | ||
skip_unpack: true | ||
|
||
# This step is to check what files are downloaded and how they are structured, as well as binary sizes for releases | ||
- name: List files | ||
run: | | ||
ls -Rlh | ||
|
||
# Create new release draft | ||
- name: Create release | ||
id: create_release | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
run: | | ||
VERSION=${{ needs.check-versions.outputs.version }} | ||
TAG=v$VERSION | ||
ls --recursive -l | ||
gh release create $TAG CANBridge-${{ github.sha }}/* --repo $GITHUB_REPOSITORY --draft --title "Version $VERSION" | ||
uses: softprops/action-gh-release@v2 | ||
with: | ||
draft: true | ||
generate_release_notes: true | ||
tag_name: v${{ steps.get-version.outputs.version }} | ||
name: Version ${{ steps.get-version.outputs.version }} | ||
files: | | ||
**/** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this necessary with the removal of the serial driver?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not necessary as the serial driver is removed. However, building back with SocketCAN or some other framework with CAN would be beneficial to leave in. There is no hurting with keeping it here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that true if we end up using DriverKit instead of IOKit? According to this page, DriverKit is an IOKit replacement: https://developer.apple.com/system-extensions/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
USBDriverKit
certainly sounds like what we want: https://developer.apple.com/documentation/usbdriverkitThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at the transcript of the video in that first link, there's a lot of emphasis on DriverKit being better than kernel extensions, which I'd think never would have been necessary in the first place for what we're doing.
It seems like DriverKit would require a separate System Extension that would handle all direct communication with the USB device, and then CANBridge would communicate with our DriverKit driver. That would be fine, but I think we'd have to design a whole API between our driver and CANBridge, which does sound like more work.
IOKit (and specifically
IOUSBLib.h
) on the other hand looks like a model more similar to what we're doing on Windows (raw USB transactions through WinUSB). The "Important" box on this page makes me a bit nervous, but it's really not clear what it's trying to say:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just noticed this in the DriverKit presentation transcript:
So yeah, I think IOKit is probably the way to go if we don't need any kind of extra OS-level permissions like a kext would have.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So from reading about macOS drivers as a whole when I made the PR, its a mess at the moment on which framework to use.
IOKit is only available on macOS but is being given the "deprecated but not deprecated" treatment. Those drivers run as kernel extensions and because in the name of security and stability those are getting moved over to DriverKit which solves the security and stability issues. Lots of documentation for how to do things since its been around for so much longer, but Apple has other ideas on what to do.
If we want maintained support for the future, IOKit will be discouraged from developing for in favor of using DriverKit.
Could be totally misinterpreting this as there was a whole macOS update that happened that added more things to DriverKit that I haven't dived into the docs for but the writing on the wall looks to be there.