CMake install cannot find libopenblas.a when using the Xcode generator

[heyleadro]

I'm building for iOS armv8 sdk 13.04 and 14.4, macos armv8 sdk 11.1
builds via conan:

cmake -G ""Xcode"" -DCMAKE_TOOLCHAIN_FILE="conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="OpenBLAS" -DNOFORTRAN="ON" -DBUILD_WITHOUT_LAPACK="ON" -DBUILD_LAPACK_DEPRECATED="OFF" -DBUILD_TESTING="OFF" -DUSE_THREAD="ON" -DMSVC_STATIC_CRT="OFF" -DBUILD_STATIC_LIBS="ON" -DBUILD_SHARED_LIBS="OFF" -DONLY_CBLAS="1" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" "OpenBLAS"

xcodebuild -project OpenBLAS.xcodeproj build -target ALL_BUILD -configuration RelWithDebInfo -hideShellScriptEnvironment

xcpretty output for iOS 13.04:
....

Compiling ztrsm_outucopy.c
Building library libkernel.a
Running script 'CMake PostBuild Rules'
Building OpenBLAS/openblas_static [RelWithDebInfo]
Check Dependencies
Aggregate OpenBLAS/ALL_BUILD [RelWithDebInfo]
Check Dependencies
Running script 'CMake Rules'
Build Succeeded
$ cmake --install ./build --config ${BUILD_TYPE} --prefix ./install
CMake Error at build/cmake_install.cmake:80 (file):
file INSTALL cannot find
"OpenBLAS/build/lib/RELWITHDEBINFO/libopenblas.a":
No such file or directory.

output for iOS 14.4 and macos:
....

Compiling cgbmv_d.c
Compiling cgbmv_c.c
Building library libdriver_level2.a
Running script 'Generate CMakeFiles/ALL_BUILD'
Build Succeeded
$ cmake --install ./build --config ${BUILD_TYPE} --prefix ./install
CMake Error at build/cmake_install.cmake:62 (file): //! line 80 for ios 14.4
file INSTALL cannot find
"OpenBLAS/build/lib/RELWITHDEBINFO/libopenblas.a":
No such file or directory.

I mean the difference is that there's nothing about openblas_static in 14.4 and macos,
anyway libopenblas.a seems to be missing, any ideas ?
P.S. adroid, win, linux etc builds and installs successfully with almost the same options.

[martin-frbg]

Which release/version are you building ? And what .a files do you see generated in your build tree ? (The messages "building libkernel.a"/"building libdriver_level2.a" look a bit strange to me, unless these names are a feature of the xcode build that I am not familiar with. At least it looks strangely like it is creating a separate .a per source directory without even trying to collate them into a single file)

[heyleadro]

Building relwithdebinfo, repo forked ~after this #4041.
I see:
libinterface.a
libdriver_level2.a
libdriver_level3.a
libdriver_others.a
libkernel.a

For example, on android amrv8 its like:
Built target interface
Built target kernel and so on, ending with:
[ 99%] Built target kernel
[100%] Linking C static library lib/libopenblas.a
[100%] Built target openblas_static

[martin-frbg]

No idea, sorry. Could be an xcode or cmake problem on that platform - all the relevant (missing) messages are generated by either of them. OTOH, is what you copied above the entire cmake command line ? (I.e. no setting of TARGET, no cross-compilation - what is your build system here please ?)

[heyleadro]

it is entire cmake command,
xcodebuild is invoked by:

cmake --build "./build" --config RelWithDebInfo
I tried
cmake --build "./build" --config RelWithDebInfo --target openblas // xcodebuild: error: The project 'OpenBLAS.xcodeproj' does not contain a target named 'openblas'.

cmake --build "./build" --config RelWithDebInfo --target openblas_static // no change in the output

i'm cross compiling for ios on macmini x86 catalina, but macos is not. TARGET is set correctly automatically ( manually I had to set on android ). I just tried to set TARGET manually and haven't seen any changes, except on macos it is ARMV8 now instead of automatically chosen VORTEX.

P.S. about android log I attached -> it is builds and install correctly and building/linking libopenblas.a only,, as you said it is strange that on ios/macos all of them are separated

P.P.S If you have no ideas and this thread will be closed, can you please briefly tell what is correct TARGET for i486 and android x86 ? I'm setting NEHALEM and ATOM respectively and it seems to be ok for build but not tested yet.

[brada4]

x86(_64) supports TARGET=GENERIC, it means -mtune=generic in addition to your compilers' defaults that presumably targets lowest capable CPU to be encountered.

[heyleadro]

@brada4 @martin-frbg Thanks guys, should I close this thread if there are still no ideas?

[martin-frbg]

Not really familiar with either xcodebuild or conan, will have to see if I can set up a CI job that replicates this.
(I wonder if there is a way to get more information about the progress of the build, like CMAKE_VERBOSE_MAKEFILE=ON in cmake/gmake builds.)

[heyleadro]

full log is not reproducable due to machine constraints. Btw building via make is fine (at least errors already solved in issues:))

[martin-frbg]

thanks - "via make" meaning pure Makefile build, or cmake+make combo ?

[heyleadro]

cmake generation like stated above :
cmake -G ""Xcode"" -DCMAKE_TOOLCHAIN_FILE="conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="OpenBLAS" -DNOFORTRAN="ON" -DBUILD_WITHOUT_LAPACK="ON" -DBUILD_LAPACK_DEPRECATED="OFF" -DBUILD_TESTING="OFF" -DUSE_THREAD="ON" -DMSVC_STATIC_CRT="OFF" -DBUILD_STATIC_LIBS="ON" -DBUILD_SHARED_LIBS="OFF" -DONLY_CBLAS="1" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" "OpenBLAS"

than https://github.com/xianyi/OpenBLAS/wiki/How-to-build-OpenBLAS-for-iPhone-iOS-(ARMv8)
than make TARGET=ARMV8 PREFIX=/path/to/prefix/install install

[martin-frbg]

Ok, so a regular Makefile build without using any of the preparations made with cmake. I believe a cmake
run to generate makefiles instead of an xcode project should work as well, but I have no idea if the contents of conan_toolchain.cmake would be appropriate for that use.

[martin-frbg]

I wonder if you could provide your conan_toolchain.cmake and OpenBLAS.xcodeproj files ?

[heyleadro]

I'll try to extract them from vm later. But I think it's easy to generate yours with conanfile.py and call conan build

tc = CMakeToolchain(self)
tc.generate()
generates conan_toolchain.cmake in generate() method

cmake = CMake(self)
cmake.configure()
cmake.build()
in the build() method will call cmake -G "Xcode" ..... and cmake --build, which will call xcode

[martin-frbg]

Thanks - it's just that I'm not really familiar with conan, and the reproducer is intended to run as an Azure CI job

[martin-frbg]

suspecting this to be a problem in the conan files (maybe somewhat related to conan-io/conan#13677 ) but I cannot get my conanfile.py to work

[heyleadro]

from conan import ConanFile
from conan.tools.cmake import CMake, CMakeToolchain, CMakeDeps, cmake_layout
from conan.tools.files import copy
from conan.tools.scm import Version
from conan.tools.apple import XcodeBuild
from conan import tools
from os.path import join
import os
from conan.tools.gnu import Autotools

class OpenblasConan(ConanFile):
name = "openblas"
description = "An optimized BLAS library based on GotoBLAS2 1.13 BSD version"
revision_mode = "scm" # the commit ID will be used as the recipe revision
settings = "os", "compiler", "build_type", "arch"
version = "0.3.23"

options = {
    "shared": [True, False],
    "with_tests": [True, False],
    "fPIC": [True, False],
}
default_options = {
    "with_tests": False,
    "shared": False,
    "fPIC": True,
}

def layout(self):
    """
        Define folder layout for build system integrations
    """
    self.folders.build = "build"
    self.folders.generators = join(self.folders.build, "generators")
    self.cpp.build.includedirs = ["include"]
    self.cpp.build.bindirs = ["bin"]
    self.cpp.build.libdirs = ["lib"]



def generate(self):
    """
        Generate toolchain for library and configs for deps
    """
    tc = CMakeToolchain(self)
    
    tc.cache_variables["NOFORTRAN"] = True
    tc.cache_variables["BUILD_WITHOUT_LAPACK"] = True
    tc.cache_variables["BUILD_LAPACK_DEPRECATED"] = False
    tc.cache_variables["BUILD_TESTING"] = False
    tc.cache_variables["USE_THREAD"] = True
    tc.cache_variables["MSVC_STATIC_CRT"] = False
    tc.cache_variables["BUILD_STATIC_LIBS"] = True
    tc.cache_variables["BUILD_SHARED_LIBS"] = False
    tc.cache_variables["ONLY_CBLAS"] = 1
    
    if self.settings.os == "iOS":
        if self.settings.arch == "armv8":
            tc.cache_variables["TARGET"] = "ARMV8"

    if self.settings.os == "Macos" and self.settings.arch == "armv8":
        tc.cache_variables["TARGET"] = "ARMV8"



    tc.generate()

    deps = CMakeDeps(self)
    deps.generate()

def build(self):
    """
        Configure cmake and build the library
    """
    if self.settings.os not in ["iOS", "Macos"]:
        cmake = CMake(self)
        cmake.configure()
        cmake.build()
        self.run("cmake --install ../build --config ${BUILD_TYPE} --prefix ../install")
    else:
        self.run("make -C ../ DYNAMIC_ARCH=0 NO_LAPACK=1 BUILD_RELAPACK=0 ONLY_CBLAS=1 NOFORTRAN=1 NO_FORTRAN=1 TARGET=ARMV8 NO_SHARED=ON BUILD_WITHOUT_LAPACK=ON HOSTCC=clang NUM_THREADS=32")
        self.run("make -C ../ TARGET=ARMV8 PREFIX=./install install")

    if self.options.with_tests:
        cmake.test()

@martin-frbg

[heyleadro]

The normal way is not branching inside build method.
For example:
def build(self):
cmake = CMake(self)
cmake.configure()
cmake.build()

  The build method is called with conan build command

like: conan build . --build=missing

[heyleadro]

So conan just invokes cmake -GXcode or something with cmake.configure() inside build() using variables and toolchain generated in generate() and then xcodebuild with cmake.build()

[heyleadro]

suspecting this to be a problem in the conan files (maybe somewhat related to conan-io/conan#13677 ) but I cannot get my conanfile.py to work

Ok I will check issue later

[heyleadro]

It doesnt look like this issue you mentioned.

1. I tried to copy everything and everywhere.
2. cmake --install invokes separately.
3. package method used cmake.install() doesn't work.
4. as I remember, I tried to build with cmake without conan.
5. it just not building .a or any symlink to it.
   perhaps its just a combination of versions/sdk/os etc.

[martin-frbg]

Thank you. I have now (finally) gotten the CI job to run with your conanfile.py, but everything appears to have worked (using xcode 12.4 and sdk 14.4)

[heyleadro]

Can you show the output?
do you call cmake --install after conan build ?

[martin-frbg]

Not sure if you can access the log from the latest CI run of PR #4070 on Azure here:
https://dev.azure.com/xianyi/OpenBLAS/_build/results?buildId=4206&view=logs&jobId=7bcc6be3-ccc9-52f0-94c2-6820b1c5117c&j=7bcc6be3-ccc9-52f0-94c2-6820b1c5117c&t=2dcd8b48-a838-5575-8a8e-111cad35ff21

the job itself is essentially fetching the conanfile.py that you provided, and then running conan build . - I hope this is correct ?
(see https://github.com/xianyi/OpenBLAS/pull/4070/files - ignore the commented-out lines at the end)

[martin-frbg]

Hm, I dont have the separate cmake --installafter the build, but the build itself appears to have completed normally, including creating of the dylib from the static one. There is a slightly confusedRUN make install; make -f Makefile.install` running as part of your conanfile, and that one appears to have found the libraries too.

[heyleadro]

@martin-frbg sorry, my mistake. Please check the build() method. You see there's branching
def build(self):
"""
Configure cmake and build the library
"""
if self.settings.os not in ["iOS", "Macos"]:
cmake = CMake(self)
cmake.configure()
cmake.build()
self.run("cmake --install ../build --config ${BUILD_TYPE} --prefix ../install")
else:
self.run("make -C ../ DYNAMIC_ARCH=0 NO_LAPACK=1 BUILD_RELAPACK=0 ONLY_CBLAS=1 NOFORTRAN=1 NO_FORTRAN=1 TARGET=ARMV8 NO_SHARED=ON BUILD_WITHOUT_LAPACK=ON HOSTCC=clang NUM_THREADS=32")
self.run("make -C ../ TARGET=ARMV8 PREFIX=./install install")

#######

the ideal way is

def build(self):
"""
Configure cmake and build the library
"""
cmake = CMake(self)
cmake.configure()
cmake.build()

and ci script :

• script: |
  ""something before""
  conan build .
  cmake --install #probably some path args
  ""something after""

[heyleadro]

@martin-frbg The way you built it in pipeline you showed, I had to make changes in both conanfile and ci to make it work using make instead of cmake etc

[martin-frbg]

Modified the build in PR #4070 and it still passes (builds libopenblas.a and the subsequent cmake --install build finds it). Can you please check if I am still doing anything differently/wrong ? https://dev.azure.com/xianyi/OpenBLAS/_build/results?buildId=4229&view=logs&jobId=7bcc6be3-ccc9-52f0-94c2-6820b1c5117c&j=7bcc6be3-ccc9-52f0-94c2-6820b1c5117c&t=2dcd8b48-a838-5575-8a8e-111cad35ff21

[heyleadro]

CmdLine:

1. input profile : i guess it is targeting x86_64 macos... check getarch result). Maybe it is ok I'm just confused that it builds for ios
2. 139-140 lines: there's cmake -G "UNIX files" instead of -GXcode
3. cmake --build "/Users/runner/work/1/s/build" -- -j3. I am not sure that it is calling xcodebuild (probably its invoking make?)
   other things seems ok.

Anyway, I guess it is time-consuming enough to make conan work as it works in my environment. So if you want, you may try to comment everything inside the build() method and in ci script after conan build put commands I provided in the very first message (cmake -Gxcode, xcodebuild, cmake install) in this thread. This way you will have generated toolchain and the commands conan calls in the build method will be in ci script.
For now, we may hold this issue because I have a choice to build it successfully, so it is not significant for me at this time.
But If you want to continue exploring this problem, I will help

[martin-frbg]

Thanks. So I think I have reproduced it now (after some symlink trickery required by the CI environment). However, I am not sure what to make of it - the build appears to be using libtool to create the .a files in individual subdirectories, which suggests that the Xcode generator may be misreading the toplevel CMakeLists.txt (that the Ninja and Makefiles generators have no problems with).

[heyleadro]

the build appears to be using libtool to create the .a files in individual subdirectories

Yes, the "broken" build generates all of *.a
so you think the problem is in xcode staff?

[martin-frbg]

It does things differently than what is in the CMakelists, so it is something I do not really have control over.

[martin-frbg]

Now that I can finally try this locally, I find that XCode is doing exactly what it has been told here - as far as I can tell, the xcodeproj file (and the conan-generated cmake file before it) treats the subdirectories of OpenBLAS as individual projects to build, which is bound to fail.
So in my opinion the error is somewhere in the conanfile.py, or (perhaps less likely) Conan is not designed to interoperate with CMake in a project laid out like OpenBLAS