Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding ROOT_jll #9300

Merged
merged 30 commits into from
Feb 19, 2025
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
b618f30
Merge branch 'ROOT-1'
peremato Aug 22, 2024
a503434
Added more library products
peremato Aug 23, 2024
19cf8bf
Build the global PCH
peremato Aug 23, 2024
4231068
typo in patch
peremato Aug 23, 2024
717866f
Temporary fix for host local prefix to simplify recipe
peremato Sep 4, 2024
dc427fa
First working version.
grasph Jan 30, 2025
3ba7dfa
Fix an issue when compilation is done the on buildkite CI
grasph Feb 4, 2025
0994081
Update R/ROOT/build_tarballs.jl
grasph Feb 14, 2025
d753723
use lock_microarchitecture=false option instead of patching compiler …
grasph Feb 14, 2025
4441235
fixed compilation issue with lastly committed oneTBB_jll by setting c…
grasph Feb 14, 2025
8411b7b
change `nproc` by $nproc and reduced number of parallel jobs.
grasph Feb 15, 2025
521d5fa
Dependencies reviewed
grasph Feb 16, 2025
4ca3d9d
Add diagnostic on kill and remove LLVM link jobs limits
grasph Feb 17, 2025
c8b7c6a
Remove signal trap and change linker
grasph Feb 17, 2025
160be3e
add util_linux_jll dependencies
grasph Feb 17, 2025
f224810
rollback previous commit and switched off Davix
grasph Feb 17, 2025
5128e8c
Trying Unix Makefile to solve CI build issue
grasph Feb 17, 2025
0680bb0
Revert to ninja and try CCACHE_RECACHE
grasph Feb 17, 2025
fdd7971
export CCACHE_RECACHE
grasph Feb 17, 2025
a4e3161
Attempt CI with less output lines
grasph Feb 18, 2025
1ecdae0
Second build with a single job for debugging
grasph Feb 18, 2025
7cd0c04
Roll back to version before the CI issue in view of introducing chang…
grasph Feb 18, 2025
7583a39
remove LAPACK and OpenBLAS _jll dependencies
grasph Feb 18, 2025
8ae7955
add GLU_jll dependency
grasph Feb 18, 2025
71580c8
add GLEW_jll dependency
grasph Feb 18, 2025
3c906f2
add nlohmann_json_jll dependency
grasph Feb 18, 2025
13aedcb
Revert "add nlohmann_json_jll dependency"
grasph Feb 18, 2025
151ef4b
add CFITSIO_jll dependency
grasph Feb 18, 2025
4bf78ab
add uuid of dependencies that missed it
grasph Feb 19, 2025
7efc7f9
fix blas depedency using OpenBLAS32 since ROOT uses the PL64 interface
grasph Feb 19, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
216 changes: 216 additions & 0 deletions R/ROOT/build_tarballs.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.

using BinaryBuilder, Pkg

name = "ROOT"
version = v"6.32.8"

rootgithash = Dict(v"6.32.6" => "5380676d1f0ed3055048f305de917572f976a090",
v"6.32.8" => "a1748a069d2d5f27ec528cfd89b54b12dcba793e")


# Collection of sources required to complete build
sources = [
GitSource("https://github.com/root-project/root.git", rootgithash[version]),
DirectorySource("./bundled")
]

# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE
if echo "$target" | grep -q musl; then #build wih musl library
# Disabling what is not working with musl:
CMAKE_EXTRA_OPTS=(-Ddavix=OFF -Dxrootd=OFF -Dssl=OFF -Druntime_cxxmodules=OFF -Droofit=OFF)
else
CMAKE_EXTRA_OPTS=(-Druntime_cxxmodules=OFF)
fi

#Uncomment for a minimal build for debugging purposes
#CMAKE_EXTRA_OPTS+=(-Dclad=OFF -Dhtml=OFF -Dwebgui=OFF -Dcxxmodules=OFF -Dproof=OFF -Dtmva=OFF -Drootfit=OFF -Dxproofd=OFF -Dxrootd=OFF -Dssl=OFF -Dpyroot=OFF -Dtesting=OFF -Droot7=OFF -Dspectrum=OFF -Dunfold=OFF -Dasimage=OFF -Dgviz=OFF -Dfitiso=OFF -Dcocoa=OFF -Dopengl=OFF -Dproof=OFF -Dxml=OFF -Dgfal=OFF -Dmpi=OFF)

export SYSTEM_INCLUDE_PATH="`g++ --sysroot="/opt/$target/$target/sys-root" -E -x c++ -v /dev/null 2>&1 | awk '{gsub(\"^ \", \"\")} /End of search list/{a=0} {if(a==1){s=s d $0;d=":"}} /#include <...> search starts here/{a=1} END{print s}'`"

# build-in compilation of the libAfterImage library needs this directory
mkdir -p /tmp/user/0

cd /

cd "$WORKSPACE"

# For the cross-compiling, LLVM needs to compile the llvm-tblgen tool
# for the host. The ROOT LLVM code tree does not include the test and
# benchmark directories cmake will need to configure for the host for
# this bootstrap. Therefore, we need to disable the build of the test and
# benchmarks.
sed -i 's/\(option(LLVM_INCLUDE_TESTS[[:space:]].*[[:space:]]\)on)\(.*\)/\1OFF)\2/i
s/\(option(LLVM_INCLUDE_BENCHMARKS[[:space:]].*[[:space:]]\)on)\(.*\)/\1OFF)\2/i' \
srcdir/root/interpreter/llvm-project/llvm/CMakeLists.txt
echo "set(CXX_STANDARD 17)" >> srcdir/root/interpreter/llvm-project/llvm/CMakeLists.txt

# N llvm links. LLVM link command needs 15GB
njobs=${nproc}
LLVM_PARALLEL_LINK_JOBS=`grep MemTot /proc/meminfo | awk '{a=int($2/15100000); if(a>'"$njobs"') a='"$njobs"'; if(a<1) a=1; print a;}'`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like you need to do all of this because you were using setting njobs incorrectly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This follows the recommendations from https://llvm.org/docs/CMake.html#frequently-used-llvm-related-variables, section LLVM_PARALLEL_{COMPILE,LINK}_JOBS. Machines have often less than 15GB RAM per core, so it it is also needed without the factor 2.


# For the rootcling execution performed during the build:
echo "include_directories(SYSTEM /opt/$target/$target/sys-root/usr/include)" >> ${CMAKE_TARGET_TOOLCHAIN}

# Build with or without debug information: Debug / Release
BUILD_TYPE=Release

#Patch needed for gcc [9.1.0, 9.3.0)
cat <<EOF | gcc -x c -o need_variant_patch -
int main(){
long vers = __GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__;
return (90100 <= vers && vers < 90300) ? 0 : 1;
}
EOF

./need_variant_patch && (cd / && atomic_patch -p1 ${WORKSPACE}/srcdir/patches/$target-variant.patch)

if [ $target != $MACHTYPE ]; then #cross compilation

(cd srcdir
# patch to fix cross-compilation with rootcling
atomic_patch -p1 patches/rootcling-cross-compile_""" * string(version) * raw""".patch

# patch to fix cross-compilation for afterimage
atomic_patch -p1 patches/afterimage-cross-compile.patch
)

# Compile for the host binary used in the build process
# Davix is switched off, as otherwise build fails in buildkite CI. It should not be
# needed for the NATIVE tools.
mkdir NATIVE
cmake -GNinja \
-DCMAKE_TOOLCHAIN_FILE=${CMAKE_HOST_TOOLCHAIN} \
-DLLVM_HOST_TRIPLE=$MACHTYPE \
-DLLVM_PARALLEL_LINK_JOBS=$LLVM_PARALLEL_LINK_JOBS \
-DCXX_STANDARD=c++17 \
-DCLANG_DEFAULT_STD_CXX=cxx17 \
"${CMAKE_EXTRA_OPTS[@]}" \
-DCMAKE_BUILD_TYPE=$BUILD_TYPE -DLLVM_BUILD_TYPE=$BUILD_TYPE \
-DCLING_CXX_PATH=g++ \
-DCLING_TARGET_GLIBC=1 \
-DCLING_TARGET_GLIBCXX=1 \
-DCLING_SYSTEM_INCLUDE_PATH="$SYSTEM_INCLUDE_PATH" \
-Ddavix=OFF \
-B NATIVE -S srcdir/root

cmake --build NATIVE -- -j$njobs rootcling_stage1 rootcling llvm-tblgen clang-tblgen llvm-config llvm-symbolizer

CMAKE_EXTRA_OPTS+=($CMAKE_EXTRA_OPTS "-DNATIVE_BINARY_DIR=$PWD/NATIVE" \
"-DLLVM_TABLEGEN=$PWD/NATIVE/interpreter/llvm-project/llvm/bin/llvm-tblgen" \
"-DCLANG_TABLEGEN=$PWD/NATIVE/interpreter/llvm-project/llvm/bin/clang-tblgen" \
"-DLLVM_CONFIG_PATH=$PWD/NATIVE/interpreter/llvm-project/llvm/bin/llvm-config" \
"-DCLING_SYSTEM_INCLUDE_PATH=$SYSTEM_INCLUDE_PATH")
fi

# CPLUS_INCLUDE_PATH used to set system include path for rootcling in absence
# of a -sysroot option. It should be transparent gcc and target build as it set the path
# to the value obtained from gcc itself before setting CPLUS_INCLUDE_PATH and using
# the same sysroot option as for compilation.
export CPLUS_INCLUDE_PATH="$SYSTEM_INCLUDE_PATH"
mkdir build
cmake -GNinja \
-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
-DCMAKE_INSTALL_PREFIX=$prefix \
-DLLVM_HOST_TRIPLE=$target \
-DLLVM_PARALLEL_LINK_JOBS=$LLVM_PARALLEL_LINK_JOBS \
-DCXX_STANDARD=c++17 \
-DCLANG_DEFAULT_STD_CXX=cxx17 \
"${CMAKE_EXTRA_OPTS[@]}" \
-DCMAKE_BUILD_TYPE=$BUILD_TYPE -DLLVM_BUILD_TYPE=$BUILD_TYPE \
-DCLING_CXX_PATH=g++ \
-Dfound_urandom_EXITCODE=0 \
-Dfound_urandom_EXITCODE__TRYRUN_OUTPUT="" \
-B build -S srcdir/root

# Build the code
cmake --build build -j${njobs}

# Install the binaries
cmake --install build --prefix $prefix
install -Dvm 755 build/core/rootcling_stage1/src/rootcling_stage1 -t "${bindir}"
"""

open("recipe.sh", "w") do io
println(io, script)
end

# Add to the recipe script commands to write the recipe in a file into the sandbox
# to ease debugging with the --debug build_tarballs.jl option.
scriptwrapper = """
cat > "\$WORKSPACE/recipe.sh" <<END_OF_SCRIPT
$(replace(script, "\\" => "\\\\", "\$" => "\\\$", "`" => "\\`"))
END_OF_SCRIPT

chmod a+x "\$WORKSPACE/recipe.sh"
$script
"""

# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
platforms = [
# Platform("i686", "linux"; libc = "glibc"),
Platform("x86_64", "linux"; libc = "glibc"),
# Platform("aarch64", "linux"; libc = "glibc"),
# Platform("armv6l", "linux"; call_abi = "eabihf", libc = "glibc"),
# Platform("armv7l", "linux"; call_abi = "eabihf", libc = "glibc"),
# Platform("powerpc64le", "linux"; libc = "glibc"),
# Platform("i686", "linux"; libc = "musl"),
# Platform("x86_64", "linux"; libc = "musl"),
# Platform("aarch64", "linux"; libc = "musl"),
# Platform("armv6l", "linux"; call_abi = "eabihf", libc = "musl"),
# Platform("armv7l", "linux"; call_abi = "eabihf", libc = "musl")
]


# The products that we will ensure are always built
products = Product[
ExecutableProduct("root", :root)
ExecutableProduct("rootcling", :rootcling)
ExecutableProduct("rootcling_stage1", :rootcling_stage1)
]

# Dependencies that must be installed before this package can be built
dependencies = [
#Mandatory dependencies
BuildDependency("Xorg_xorgproto_jll")
Dependency("Xorg_libX11_jll")
Dependency("Xorg_libXpm_jll")
Dependency("Xorg_libXft_jll")

#Optionnal dependencies (if absent, either a feature will be disabled or a built-in version will be compiled)
Dependency("VDT_jll")
Dependency("XRootD_jll")
Dependency(PackageSpec(name="LAPACK_jll", uuid="51474c39-65e3-53ba-86ba-03b1b862ec14"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need both openblas and lapack?

Dependency("Lz4_jll")
Dependency(PackageSpec(name="FFTW_jll", uuid="f5851436-0d7a-5f13-b9de-f02708fd171a"))
Dependency(PackageSpec(name="Giflib_jll", uuid="59f7168a-df46-5410-90c8-f2779963d0ec"))
Dependency(PackageSpec(name="Zstd_jll", uuid="3161d3a3-bdf6-5164-811a-617609db77b4"))
Dependency(PackageSpec(name="PCRE2_jll", uuid="efcefdf7-47ab-520b-bdef-62a2eaa19f15"))
Dependency(PackageSpec(name="Graphviz_jll", uuid="3c863552-8265-54e4-a6dc-903eb78fde85"))
Dependency(PackageSpec(name="xxHash_jll", uuid="5fdcd639-92d1-5a06-bf6b-28f2061df1a9"))
Dependency("XZ_jll")
Dependency(PackageSpec(name="Librsvg_jll", uuid="925c91fb-5dd6-59dd-8e8c-345e74382d89"))
Dependency("FreeType2_jll")
Dependency("Xorg_libICE_jll")
Dependency("Xorg_libSM_jll")
Dependency("Xorg_libXfixes_jll")
Dependency("Xorg_libXi_jll")
Dependency("Xorg_libXinerama_jll")
Dependency("Xorg_libXmu_jll")
Dependency("Xorg_libXt_jll")
Dependency("Xorg_libXtst_jll")
Dependency("Xorg_xcb_util_jll")
Dependency("Xorg_libxkbfile_jll")
Dependency("Libglvnd_jll")
Dependency("OpenBLAS_jll")
Dependency("oneTBB_jll", compat="2021.9.0")
]

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, scriptwrapper, platforms, products,
dependencies; julia_compat="1.6", preferred_gcc_version=v"9", lock_microarchitecture=false)

10 changes: 10 additions & 0 deletions R/ROOT/bundled/patches/afterimage-cross-compile.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
--- a/root/cmake/modules/SearchInstalledSoftware.cmake 2024-06-28 12:55:26.471599664 +0200
+++ b/root/cmake/modules/SearchInstalledSoftware.cmake 2024-06-28 12:57:58.417875621 +0200
@@ -499,6 +499,7 @@
DOWNLOAD_COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_SOURCE_DIR}/graf2d/asimage/src/libAfterImage AFTERIMAGE
INSTALL_DIR ${CMAKE_BINARY_DIR}
CONFIGURE_COMMAND ./configure --prefix <INSTALL_DIR>
+ --host=$ENV{target}
--libdir=<INSTALL_DIR>/lib
--with-ttf ${_ttf_include} --with-afterbase=no
--without-svg --disable-glx ${_after_mmx}
Loading