Skip to content

Commit

Permalink
Merge gh-actions branch into main
Browse files Browse the repository at this point in the history
main..gh-actions:
- CI/CD: Fix release permissions
- Fix macos packaging on debian
- CI/CD: Remove unwanted tools
- CI/CD: Attempt to fix OOM killing GH runner during LTO
- CI/CD: Fix glibc errors
- CI/CD: Downgrade to ubuntu-20.04
- CI/CD: Use target os matrix & fix release
- CI/CD: Remove Windows (temporarily)
- CI/CD: Move to AdityaGarg8/[email protected]
- CI/CD: Limit CPU as well
- CI/CD: Create cgroup with 80% mem limit
- CI/CD: Revert "Remove .mozbuild caching"
- CI/CD: Expand root & swap build space
- CI/CD: Remove .mozbuild caching
- CI/CD: Check disk space after checkout
- CI/CD: Use easimon/maximize-build-space
- CI/CD: Add workflow dispatch trigger
- CI/CD: Experimental fix for clang, add swap space
- CI/CD: Save .mozbuid cache after Build
  • Loading branch information
daijro committed Aug 16, 2024
1 parent 33137af commit 1d31bad
Show file tree
Hide file tree
Showing 3 changed files with 85 additions and 38 deletions.
109 changes: 80 additions & 29 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,46 @@ on:
tags:
- "*"

env:
MOZBUILD_PATH: ${{ github.workspace }}/.mozbuild

jobs:
build:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
strategy:
matrix:
target: [linux, windows, macos]

steps:
- name: Maximize build space
uses: AdityaGarg8/[email protected]
with:
remove-dotnet: "true"
remove-android: "true"
remove-haskell: "true"
remove-codeql: "true"
remove-docker-images: "true"
remove-cached-tools: "true"
remove-swapfile: "true"
verbose: "false"

- name: Remove unwanted tools
# Originally from here: https://github.com/AdityaGarg8/remove-unwanted-software/blob/master/action.yml
# Modified to keep llvm.
run: |
sudo apt-get remove -y '^aspnetcore-.*' > /dev/null
sudo apt-get remove -y '^dotnet-.*' > /dev/null
sudo apt-get remove -y 'php.*' > /dev/null
sudo apt-get remove -y '^mongodb-.*' > /dev/null
sudo apt-get remove -y '^mysql-.*' > /dev/null
sudo apt-get remove -y azure-cli google-chrome-stable firefox ${POWERSHELL} mono-devel libgl1-mesa-dri --fix-missing > /dev/null
if [[ (${CODENAME} = focal) || (${CODENAME} = jammy) ]]; then
sudo apt-get remove -y google-cloud-sdk --fix-missing > /dev/null
sudo apt-get remove -y google-cloud-cli --fix-missing > /dev/null
fi
sudo apt-get autoremove -y > /dev/null
sudo apt-get clean > /dev/null
- uses: actions/checkout@v2
with:
fetch-depth: 1

- name: Set up Go
uses: actions/setup-go@v2
Expand All @@ -26,6 +57,15 @@ jobs:
with:
python-version: "3.11"

- name: Set up Clang
uses: egor-tensin/setup-clang@v1
with:
version: latest
platform: x64

- name: Check disk space
run: df -h

- name: Install dependencies
run: |
sudo apt-get update
Expand All @@ -34,50 +74,61 @@ jobs:
- name: Restore .mozbuild cache
uses: actions/cache/restore@v3
with:
path: ${{ env.MOZBUILD_PATH }}
key: ${{ runner.os }}-mozbuild-${{ hashFiles('**/Makefile') }}
restore-keys: |
${{ runner.os }}-mozbuild-
path: ~/.mozbuild
key: mozbuild-${{ matrix.target }}

- name: Setup and bootstrap
run: |
make setup-minimal
make mozbootstrap
mkdir -p dist
- name: List .mozbuild contents
run: |
ls -R ~/.mozbuild
- name: Create swap space
run: |
sudo fallocate -l 16G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
free -h
- name: Build
run: python3 ./multibuild.py --target ${{ matrix.target }} --arch x86_64

- name: Save .mozbuild cache
uses: actions/cache/save@v3
with:
path: ${{ env.MOZBUILD_PATH }}
key: ${{ runner.os }}-mozbuild-${{ hashFiles('**/Makefile') }}

- name: Build
run: |
python3 ./multibuild.py --target linux windows macos --arch x86_64
path: ~/.mozbuild
key: mozbuild-${{ matrix.target }}

- name: Upload artifacts
uses: actions/upload-artifact@v2
with:
name: CamoufoxBuilds
name: CamoufoxBuilds-${{ matrix.target }}
path: dist/*

release:
needs: build
permissions:
contents: write
runs-on: ubuntu-latest

steps:
- name: Download all artifacts
uses: actions/download-artifact@v2
with:
path: artifacts

- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
uses: softprops/action-gh-release@v1
if: startsWith(github.ref, 'refs/tags/')
with:
tag_name: ${{ github.ref }}
release_name: Release ${{ github.ref }}
files: artifacts/**/*
generate_release_notes: true
draft: true
prerelease: false

- name: Upload Release Assets
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./dist/*
asset_name: CamoufoxBuilds
asset_content_type: application/zip
2 changes: 2 additions & 0 deletions assets/base.mozconfig
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,5 @@ mk_add_options MOZ_SERVICES_HEALTHREPORT=0
mk_add_options MOZ_TELEMETRY_REPORTING=0
mk_add_options MOZ_INSTALLER=0
mk_add_options MOZ_AUTOMATION_INSTALLER=0

ac_add_options --enable-lto=cross
12 changes: 3 additions & 9 deletions scripts/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,20 @@
import argparse
import glob
import os
import shlex
import shutil
import sys
import tempfile
from shlex import join

from _mixin import find_src_dir, get_moz_target, list_files, run, temp_cd

UNNEEDED_PATHS = {'uninstall', 'pingsender.exe', 'pingsender', 'vaapitest', 'glxtest'}


def run_command(command):
"""Execute a command with subprocess"""
cmd = ' '.join(shlex.quote(arg) for arg in command)
run(cmd)


def add_includes_to_package(package_file, includes, fonts, new_file, target):
with tempfile.TemporaryDirectory() as temp_dir:
# Extract package
run_command(['7z', 'x', package_file, f'-o{temp_dir}'])
run(join(['7z', 'x', package_file, f'-o{temp_dir}']), exit_on_fail=False)
# Delete package_file
os.remove(package_file)
if package_file.endswith('.tar.bz2'):
Expand Down Expand Up @@ -105,7 +99,7 @@ def add_includes_to_package(package_file, includes, fonts, new_file, target):
os.remove(os.path.join(target_dir, path))

# Update package
run_command(['7z', 'u', new_file, f'{temp_dir}/*', '-r', '-mx=9'])
run(join(['7z', 'u', new_file, f'{temp_dir}/*', '-r', '-mx=9']))


def get_args():
Expand Down

0 comments on commit 1d31bad

Please sign in to comment.