[vcpkg-ci-cmake-toolchain-file] Validate toolchain file #43527

dg0yt · 2025-01-29T08:19:01Z

WIP. Exposing toolchain bug for android, seen in #43504, #41905.

Neumann-A · 2025-01-29T09:49:10Z

What exactly is the mess? Seems like android is missing it up?

dg0yt · 2025-01-29T11:29:39Z

I don't think android will take the blame. The combination of vcpkg's chosen chainloaded toolchain together with vcpgk.cmake's guiding of find_<foo> make CMAKE find library libc++ from the NDK host binaries. The android arm triplets finally raise the error because you can't link an x64 linux lib into an arm executable.

This:

vcpkg/scripts/buildsystems/vcpkg.cmake

Lines 435 to 452 in fc97d3e

    
           z_vcpkg_add_vcpkg_to_cmake_path(CMAKE_PREFIX_PATH "") 
        
           z_vcpkg_add_vcpkg_to_cmake_path(CMAKE_LIBRARY_PATH "/lib/manual-link") 
        
           z_vcpkg_add_vcpkg_to_cmake_path(CMAKE_FIND_ROOT_PATH "") 
        
           if(NOT VCPKG_PREFER_SYSTEM_LIBS) 
        
               set(CMAKE_FIND_FRAMEWORK "LAST") # we assume that frameworks are usually system-wide libs, not vcpkg-built 
        
               set(CMAKE_FIND_APPBUNDLE "LAST") # we assume that appbundles are usually system-wide libs, not vcpkg-built 
        
           endif() 
        
           # If one CMAKE_FIND_ROOT_PATH_MODE_* variables is set to ONLY, to  make sure that ${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET} 
        
           # and ${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}/debug are searched, it is not sufficient to just add them to CMAKE_FIND_ROOT_PATH, 
        
           # as CMAKE_FIND_ROOT_PATH specify "one or more directories to be prepended to all other search directories", so to make sure that 
        
           # the libraries are searched as they are, it is necessary to add "/" to the CMAKE_PREFIX_PATH 
        
           if(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE STREQUAL "ONLY" OR 
        
              CMAKE_FIND_ROOT_PATH_MODE_LIBRARY STREQUAL "ONLY" OR 
        
              CMAKE_FIND_ROOT_PATH_MODE_PACKAGE STREQUAL "ONLY") 
        
              list(APPEND CMAKE_PREFIX_PATH "/") 
        
           endif()

leads to that:

CMake Debug Log at CMakeLists.txt:18 (find_library):
  find_library called with the following settings:

    VAR: c___LIBRARY
    NAMES: "c++"
    Documentation: Path to a library.
    Framework
      Only Search Frameworks: 0
      Search Frameworks Last: 1
      Search Frameworks First: 0
    AppBundle
      Only Search AppBundle: 0
      Search AppBundle Last: 1
      Search AppBundle First: 0
    CMAKE_FIND_USE_CMAKE_PATH: 1
    CMAKE_FIND_USE_CMAKE_ENVIRONMENT_PATH: 1
    CMAKE_FIND_USE_SYSTEM_ENVIRONMENT_PATH: 1
    CMAKE_FIND_USE_CMAKE_SYSTEM_PATH: 1
    CMAKE_FIND_USE_INSTALL_PREFIX: 1

  find_library considered the following locations:

    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/mnt/vcpkg-ci/installed/arm64-android/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/mnt/vcpkg-ci/installed/arm64-android/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/mnt/vcpkg-ci/installed/arm64-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/manual-link/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/mnt/vcpkg-ci/installed/arm64-android/lib/manual-link/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/opt/microsoft/powershell/7/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/local/sbin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/local/bin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/sbin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/bin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/sbin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/bin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/vcpkg/downloads/tools/ninja-1.12.1-linux/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/lib/aarch64-linux-android/21/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/lib/aarch64-linux-android/21/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/lib/aarch64-linux-android/21/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/local/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/local/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/local/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/vcpkg/downloads/tools/cmake-3.30.1-linux/cmake-3.30.1-linux-x86_64/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/vcpkg/downloads/tools/cmake-3.30.1-linux/cmake-3.30.1-linux-x86_64/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/vcpkg/downloads/tools/cmake-3.30.1-linux/cmake-3.30.1-linux-x86_64/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/mnt/vcpkg-ci/p/vcpkg-ci-cmake-toolchain-file_arm64-android/debug/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/mnt/vcpkg-ci/p/vcpkg-ci-cmake-toolchain-file_arm64-android/debug/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/mnt/vcpkg-ci/p/vcpkg-ci-cmake-toolchain-file_arm64-android/debug/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/X11R6/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/X11R6/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/X11R6/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/pkg/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/pkg/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/pkg/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/opt/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/opt/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/opt/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/usr/lib/X11/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/debug/lib/manual-link/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/lib/manual-link/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/opt/microsoft/powershell/7/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/local/sbin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/local/bin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/sbin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/bin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/sbin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/bin/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/vcpkg/downloads/tools/ninja-1.12.1-linux/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/lib/aarch64-linux-android/21/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/lib/aarch64-linux-android/21/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/lib/aarch64-linux-android/21/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/local/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/local/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/local/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/vcpkg/downloads/tools/cmake-3.30.1-linux/cmake-3.30.1-linux-x86_64/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/vcpkg/downloads/tools/cmake-3.30.1-linux/cmake-3.30.1-linux-x86_64/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/vcpkg/downloads/tools/cmake-3.30.1-linux/cmake-3.30.1-linux-x86_64/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/mnt/vcpkg-ci/p/vcpkg-ci-cmake-toolchain-file_arm64-android/debug/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/mnt/vcpkg-ci/p/vcpkg-ci-cmake-toolchain-file_arm64-android/debug/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/mnt/vcpkg-ci/p/vcpkg-ci-cmake-toolchain-file_arm64-android/debug/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/X11R6/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/X11R6/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/X11R6/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/pkg/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/pkg/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/pkg/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/opt/lib/aarch64-linux-android/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/opt/lib/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/opt/libc++(\.so|\.a)
    /mnt/vcpkg-ci/installed/arm64-android/usr/lib/X11/libc++(\.so|\.a)
    /android-ndk-r27c/mnt/vcpkg-ci/installed/arm64-android/debug/lib/aarch64-linux-android/libc++(\.so|\.a)
    /android-ndk-r27c/mnt/vcpkg-ci/installed/arm64-android/debug/lib/libc++(\.so|\.a)
    /android-ndk-r27c/mnt/vcpkg-ci/installed/arm64-android/debug/libc++(\.so|\.a)
    /android-ndk-r27c/mnt/vcpkg-ci/installed/arm64-android/lib/aarch64-linux-android/libc++(\.so|\.a)
    /android-ndk-r27c/mnt/vcpkg-ci/installed/arm64-android/lib/libc++(\.so|\.a)
    /android-ndk-r27c/mnt/vcpkg-ci/installed/arm64-android/libc++(\.so|\.a)
    /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/aarch64-linux-android/libc++(\.so|\.a)

  The item was found at

    /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/libc++.so

leads to that:

: && /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=aarch64-none-linux-android21 --sysroot=/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security  -fPIC   -fno-limit-debug-info -static-libstdc++ -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--no-undefined-version -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments CMakeFiles/main.dir/main.c.o -o main  /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/libc++.so  /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/21/libm.so  -latomic -lm && :
ld.lld: error: /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/libc++.so is incompatible with aarch64linux

NB: c++ shouldn't be found in default directories. It lives in implicit directories, i.e.

-- Looking up implicit C++ link libraries
C++ implicit link directories:
    /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/lib/clang/18/lib/linux/aarch64
    /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/21
    /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android
    /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib

IMHO there should be much less directories in the default path.

Neumann-A · 2025-01-29T11:59:17Z

why is /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64 a valid search path ?

dg0yt · 2025-01-29T19:46:16Z

why is /android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64 a valid search path ?

The NDK's toolchain file (recommended = "legacy") puts the NDK dir into CMAKE_PREFIX_PATH.

The test port comes with an independent CMake project. Building the test project with plain NDK and official CMake toolchain instructions shows the same issue. So yes, it is a NDK issue.

But vcpkg might want a mitigation now.

Neumann-A · 2025-01-29T22:55:26Z

https://cmake.org/cmake/help/latest/variable/CMAKE_IGNORE_PREFIX_PATH.html ?

dg0yt · 2025-01-30T04:58:44Z

https://cmake.org/cmake/help/latest/variable/CMAKE_IGNORE_PREFIX_PATH.html ?

This would also change find_program behavior, globally, But finding host programs is usually desired.

dg0yt · 2025-01-30T06:00:15Z

Well, with /lib appended, it could work for linux.

list(APPEND CMAKE_IGNORE_PREFIX_PATH "${ANDROID_TOOLCHAIN_ROOT}/lib")

dg0yt · 2025-01-30T06:02:37Z

BTW there is also a wrong suffix order for static CRT linkage.

dg0yt · 2025-02-01T09:00:18Z

BTW there is also a wrong suffix order for static CRT linkage.

This isn't easy to change. (CMAKE_FIND_LIBRARY_SUFFIXES is set after the toolchain file. Sigh.) And changing it doesn't help for Android because the dir with the shared libs is visited before the dir with static libs. (The suffixes are handled with pattern matching, unlike NAMES, which require NAMES_PER_DIR for that behavior. Sigh.)
🙈

BillyONeal · 2025-02-06T00:41:56Z

Seems reasonable, thank you!

)

[vcpkg-ci-cmake-toolchain-file] Validate toolchain file

ffd8238

dg0yt closed this Jan 30, 2025

dg0yt reopened this Jan 30, 2025

dg0yt marked this pull request as ready for review February 2, 2025 08:06

dg0yt added 3 commits February 2, 2025 09:45

Elaborate test port

d12144c

[android.cmake] Ignore host libs

d9b5e1b

Fixup

cb49706

dg0yt force-pushed the cmake-toolchain-file branch from 33c2f82 to cb49706 Compare February 2, 2025 08:45

LilyWangLL assigned MonicaLiu0311 Feb 5, 2025

MonicaLiu0311 added the category:infrastructure Pertaining to the CI/Testing infrastrucutre label Feb 5, 2025

MonicaLiu0311 approved these changes Feb 5, 2025

View reviewed changes

MonicaLiu0311 added the info:reviewed Pull Request changes follow basic guidelines label Feb 5, 2025

BillyONeal merged commit 8143d2e into microsoft:master Feb 6, 2025
17 checks passed

dg0yt deleted the cmake-toolchain-file branch February 6, 2025 02:05

autoantwort pushed a commit to autoantwort/vcpkg that referenced this pull request Feb 8, 2025

[vcpkg-ci-cmake-toolchain-file] Validate toolchain file (microsoft#43527

22bfad0

)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vcpkg-ci-cmake-toolchain-file] Validate toolchain file #43527

[vcpkg-ci-cmake-toolchain-file] Validate toolchain file #43527

dg0yt commented Jan 29, 2025

Neumann-A commented Jan 29, 2025

dg0yt commented Jan 29, 2025

Neumann-A commented Jan 29, 2025

dg0yt commented Jan 29, 2025

Neumann-A commented Jan 29, 2025

dg0yt commented Jan 30, 2025

dg0yt commented Jan 30, 2025

dg0yt commented Jan 30, 2025

dg0yt commented Feb 1, 2025 •

edited

Loading

BillyONeal commented Feb 6, 2025

[vcpkg-ci-cmake-toolchain-file] Validate toolchain file #43527

[vcpkg-ci-cmake-toolchain-file] Validate toolchain file #43527

Conversation

dg0yt commented Jan 29, 2025

Neumann-A commented Jan 29, 2025

dg0yt commented Jan 29, 2025

Neumann-A commented Jan 29, 2025

dg0yt commented Jan 29, 2025

Neumann-A commented Jan 29, 2025

dg0yt commented Jan 30, 2025

dg0yt commented Jan 30, 2025

dg0yt commented Jan 30, 2025

dg0yt commented Feb 1, 2025 • edited Loading

BillyONeal commented Feb 6, 2025

dg0yt commented Feb 1, 2025 •

edited

Loading