Issue installing hdf5r on MacBook Pro with M3 Pro chip (only when using renv)

[lucygarner]

Hi,

I have been helping a colleague to set up their R environments on a new MacBook Pro with M3 Pro chip. However, we are having difficulties installing hdf5r when using renv. hdf5r installs fine into the "System Library" (outside of renv).

However, inside an RStudio project with renv, we get the following initial error:

In file included from const_export.c:23: In file included from ./const_export.h:22: ./global.h:23:10: fatal error: 'hdf5.h' file not found #include <hdf5.h> ^~~~~~~~ 1 error generated. make: *** [const_export.o] Error 1 ERROR: compilation failed for package ‘hdf5r’ * removing ‘/Users/user1/Documents/R/new_project/renv/library/R-4.3/aarch64-apple-darwin20/hdf5r’

This error was fixed using the following command, as discussed hhoeflin/hdf5r#171:

install.packages("hdf5r", configure.args = c("--with-hdf5=/opt/homebrew/bin/h5cc"))

However, we then have the following error:

ld: library 'hdf5_hl' not found clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [hdf5r.so] Error 1 ERROR: compilation failed for package ‘hdf5r’ * removing ‘/Users/user1/Documents/R/new_project/renv/library/R-4.3/aarch64-apple-darwin20/hdf5r’

Do you have any idea why we are specifically getting these errors when in an RStudio project using renv? And how we might fix them?

Best wishes,
Lucy

[kevinushey]

To confirm -- when the package is installed outside of renv, is the package still being installed from sources? Or is a binary copy of the package being installed?

My best guess is that the PATH might differ between what you see in an renv project, and what you see in a regular R session outside of renv. Can compare what:

writeLines(strsplit(Sys.getenv("PATH"), .Platform$path.sep)[[1]])

reports in each case? I ask because I think the hdf5r package includes a configure script that tries to find some of these binaries and tools on the PATH, and so the issue may be related.

It may also be helpful to see the full output produced during package installation (so I can see what library paths are included, and what seems to be missing)

[agreco92]

I have the same issue on my Macbook pro M1 (OS: Monterrey). My R version is 4.3 (x86-84), handled using Rswitch. Did anyone solve this issue?

[JonathanNoonan]

Hey @lucygarner - did you ever get to the bottom of this? Im now having the same problem

Thanks!

[lucygarner]

Unfortunately my colleague went on extended leave, so I wasn't able to work any further on it. Sorry I can't be of more help!

[kevinushey]

Can you provide the whole output log? It should also include something like:

Using variables:
HDF5_LIBS= -L. -lhdf5_hl -lhdf5 -lz -lm
HDF5_CFLAGS=
HDF5_CPPFLAGS=

I think the problem here is that rhdf5 is trying to figure out the library paths with an invocation like:

HDF5_LIBS=`${H5CC} -show | ${GREP} -o " \-[lL][^[:space:]]\+" | tr "\n" " "`

But that's not actually supported here?

$ /opt/homebrew/bin/h5cc -show
/opt/homebrew/h5cc
dir is /opt/homebrew
clang: error: unknown argument: '-show'

It seems like it would be possible to glean the appropriate library paths from h5cc -showconfig though, e.g.

$ /opt/homebrew/bin/h5cc -showconfig
/opt/homebrew/h5cc
dir is /opt/homebrew
        SUMMARY OF THE HDF5 CONFIGURATION
        =================================

General Information:
-------------------
                   HDF5 Version: 1.14.4.3
                  Configured on: 2024-05-22
                  Configured by: Unix Makefiles
                    Host system: Darwin-23.4.0
              Uname information: Darwin
                       Byte sex: little-endian
             Installation point: /opt/homebrew/Cellar/hdf5/1.14.4.3

[kevinushey]

I filed an issue at hhoeflin/hdf5r#231, but this may actually be an issue with the hdf5 bottle distributed by Homebrew. I'm going to dig a bit more.

[JonathanNoonan]

Hi Kevin,

Thanks for looking into it. If you get any updates please let us know. I ended up scrapping what i'm doing for now and am working outside of renv in the short term for this specific project. When I return to it, which I will ultimately need to, I'll send the logs. I would assume this is impacting lots of people in the bioinformatics sphere.