Crosscompiled artefacts are wrongly named

[AlexisTM]

The Pybind11 script is using system Python to get the PYTHON_MODULE_EXTENSION (aka SO_SUFFIX) instead of using the sysroot defined in the toolchain.

This causes properly built files (aarch64) to be incorrectly named (x86_64).

$ file _rclpy_pybind11.cpython-39-x86_64-linux-gnu.so
_rclpy_pybind11.cpython-39-x86_64-linux-gnu.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=598c9e00d1b689d36fac5095805f66af558a2a82, stripped

The cause is in Pybind11, where the module extension value is found from executing Python on the system instead of the sysroot target.

  execute_process(
    COMMAND
      "${${_Python}_EXECUTABLE}" "-c"
      "import sys, importlib; s = importlib.import_module('distutils.sysconfig' if sys.version_info < (3, 10) else 'sysconfig'); print(s.get_config_var('EXT_SUFFIX') or s.get_config_var('SO'))"
    OUTPUT_VARIABLE _PYTHON_MODULE_EXTENSION
    ERROR_VARIABLE _PYTHON_MODULE_EXTENSION_ERR
    OUTPUT_STRIP_TRAILING_WHITESPACE)

We are currently on galactic, which points to pybind11_vendor#foxy.

Sadly, the quickfix only works for rclpy which explicitly calls Pybind11.

Quickfix

set(PYTHONLIBS_FOUND TRUE)
set(PYTHON_MODULE_EXTENSION cpython-39-aarch64-linux-gnu.so) # Taken by executing the python command within the sysroot (aka target)

Output with Quickfix

./rcl_interfaces/rcl_interfaces_s__rosidl_typesupport_c.cpython-39-x86_64-linux-gnu.so
./rcl_interfaces/rcl_interfaces_s__rosidl_typesupport_introspection_c.cpython-39-x86_64-linux-gnu.so
./rcl_interfaces/rcl_interfaces_s__rosidl_typesupport_fastrtps_c.cpython-39-x86_64-linux-gnu.so
./rmw_dds_common/rmw_dds_common_s__rosidl_typesupport_introspection_c.cpython-39-x86_64-linux-gnu.so
./rmw_dds_common/rmw_dds_common_s__rosidl_typesupport_c.cpython-39-x86_64-linux-gnu.so
./rmw_dds_common/rmw_dds_common_s__rosidl_typesupport_fastrtps_c.cpython-39-x86_64-linux-gnu.so
./rosgraph_msgs/rosgraph_msgs_s__rosidl_typesupport_c.cpython-39-x86_64-linux-gnu.so
./rosgraph_msgs/rosgraph_msgs_s__rosidl_typesupport_fastrtps_c.cpython-39-x86_64-linux-gnu.so
./rosgraph_msgs/rosgraph_msgs_s__rosidl_typesupport_introspection_c.cpython-39-x86_64-linux-gnu.so
./test_msgs/test_msgs_s__rosidl_typesupport_fastrtps_c.cpython-39-x86_64-linux-gnu.so
./test_msgs/test_msgs_s__rosidl_typesupport_c.cpython-39-x86_64-linux-gnu.so
./test_msgs/test_msgs_s__rosidl_typesupport_introspection_c.cpython-39-x86_64-linux-gnu.so
./action_msgs/action_msgs_s__rosidl_typesupport_fastrtps_c.cpython-39-x86_64-linux-gnu.so
./action_msgs/action_msgs_s__rosidl_typesupport_introspection_c.cpython-39-x86_64-linux-gnu.so
./action_msgs/action_msgs_s__rosidl_typesupport_c.cpython-39-x86_64-linux-gnu.so
./unique_identifier_msgs/unique_identifier_msgs_s__rosidl_typesupport_fastrtps_c.cpython-39-x86_64-linux-gnu.so
./unique_identifier_msgs/unique_identifier_msgs_s__rosidl_typesupport_introspection_c.cpython-39-x86_64-linux-gnu.so
./unique_identifier_msgs/unique_identifier_msgs_s__rosidl_typesupport_c.cpython-39-x86_64-linux-gnu.so
./builtin_interfaces/builtin_interfaces_s__rosidl_typesupport_c.cpython-39-x86_64-linux-gnu.so
./builtin_interfaces/builtin_interfaces_s__rosidl_typesupport_fastrtps_c.cpython-39-x86_64-linux-gnu.so
./builtin_interfaces/builtin_interfaces_s__rosidl_typesupport_introspection_c.cpython-39-x86_64-linux-gnu.so
./rclpy/_rclpy_pybind11cpython-39-aarch64-linux-gnu.so
./rclpy/_rclpy_signal_handlercpython-39-aarch64-linux-gnu.so

What is the clean way to make pybind11 work with crosscompilation?

[AlexisTM]

As for the other names, there is a pybin11-inspired methodology replicated in the python_cmake_module/cmake/Modules/FindPythonExtras.cmake, also executing the python script on host instead of target.

[clalancette]

In short, we don't regularly test cross-compilation (particularly with Python), so it isn't entirely surprising that it doesn't work. If you figure out ways to improve it, we would be happy to review pull requests for that.

[AlexisTM]

I fully understand that, as the buildfarms runs on target :)

The solution I would propose is to use qemu-static if a sysroot is defined. If qemu-static is not installed, then I would suggest returning with an error, asking to either install qemu-static or define the following variables.

In the meantime, if someone else bumps into it, you can add the following in the toolchain.

# See: https://github.com/ros2/pybind11_vendor to have pybind11 version, then
# See: https://github.com/pybind/pybind11/blob/8a4bca8216f94f64d9582997db5e7eaf2e4660b3/tools/pybind11NewTools.cmake#L98-L111
# Note the point!
set(PYTHON_MODULE_EXTENSION ".cpython-39-aarch64-linux-gnu.so")
# python_cmake_module workaround, PYTHON_MODULE_EXTENSION is set to PYTHON_SOABI
# See: https://github.com/ros2/python_cmake_module/blob/d31a0b13225786e076e537a6f1ba67ba5cb2e620/cmake/Modules/FindPythonExtra.cmake#L159-L197
# Note the lack of point & .so
set(PYTHON_SOABI cpython-39-aarch64-linux-gnu)

[AlexisTM]

This is "solvable" with the pull request located in ros2/python_cmake_module#12.

The goal of that PR being to stop the compilation with a nice error message telling you to define PYTHON_SOABI when cross-compiling. Plus, it would also define PYTHON_MODULE_EXTENSION as it is required in certain cases (rclpy)

[mosfet80]

try this .. rolling...mosfet80:pybind11_vendor:UpdatePybind11

i have update pybind to version 2.10.4