Installing on windows #21

wolski · 2018-10-02T14:57:12Z

Is it possible to get the package installed on MS Windows . I did run into the following error:

> devtools::install_github("nfultz/grpc")
Downloading GitHub repo nfultz/grpc@master
from URL https://api.github.com/repos/nfultz/grpc/zipball/master
Installing grpc
"C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/bin/x64/R" --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  "C:/Users/wolski/AppData/Local/Temp/RtmpuwKZn5/devtools20904dbd502/nfultz-grpc-0b08d64" --library="C:/Users/wolski/Documents/R/win-library/3.5"  \
  --install-tests 

* installing *source* package 'grpc' ...
** libs
c:/Rtools/mingw_64/bin/g++ -m64 -I"C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/include" -DNDEBUG  -I"C:/Users/wolski/Documents/R/win-library/3.5/Rcpp/include"   -I"C:/swarm/workspace/External-R-3.5.1/vendor/extsoft/include"  `pkg-config --cflags grpc`   -O2 -Wall  -mtune=core2 -c RcppExports.cpp -o RcppExports.o
pkg-config: not found
c:/Rtools/mingw_64/bin/g++ -m64 -I"C:/PROGRA~1/MIE74D~1/ROPEN~1/R-35~1.1/include" -DNDEBUG  -I"C:/Users/wolski/Documents/R/win-library/3.5/Rcpp/include"   -I"C:/swarm/workspace/External-R-3.5.1/vendor/extsoft/include"  `pkg-config --cflags grpc`   -O2 -Wall  -mtune=core2 -c client.cpp -o client.o
pkg-config: not found
client.cpp:2:23: fatal error: grpc/grpc.h: No such file or directory
 #include <grpc/grpc.h>
                       ^
compilation terminated.
make: *** [client.o] Error 1
ERROR: compilation failed for package 'grpc'
* removing 'C:/Users/wolski/Documents/R/win-library/3.5/grpc'
In R CMD INSTALL
Installation failed: Command failed (1)

The text was updated successfully, but these errors were encountered:

nfultz · 2018-10-02T14:59:54Z

It sounds possible, but I've never tried. You probably need to make a windows version of Makevars containing correct compiler options for windows..

nfultz · 2018-10-02T20:45:52Z

Not sure who sent in a winBuilder attempt, but here is the result:

* using log directory 'd:/RCompile/CRANguest/R-devel/grpc.Rcheck'
* using R Under development (unstable) (2018-09-29 r75382)
* using platform: x86_64-w64-mingw32 (64-bit)
* using session charset: ISO8859-1
* checking for file 'grpc/DESCRIPTION' ... OK
* checking extension type ... Package
* this is package 'grpc' version '0.1.0'
* package encoding: UTF-8
* checking CRAN incoming feasibility ... NOTE
Maintainer: 'Neal Fultz <[email protected]>'

New submission

Possibly mis-spelled words in DESCRIPTION:
  gRPC (3:8)

The Title field starts with the package name.
* checking package namespace information ... OK
* checking package dependencies ... ERROR
Namespace dependency not required: 'methods'

See section 'The DESCRIPTION file' in the 'Writing R Extensions'
manual.
* DONE
Status: 1 ERROR, 1 NOTE

Seems easy to fix.

wolski · 2018-10-02T20:46:40Z

Just tried it:

a) will need a grpc build with the same compiler as used by Rtools - mingw
so I did some searching and the windows prebuild binary based on mingw can be found with MSYS2.
b) Installed MSYS2 and grpc with

pacman -S mingw-w64-x86_64-grpc

c) Created Makevars.win containing:

PKG_CPPFLAGS=-IC:/msys64/mingw64/include
PKG_LIBS=-LC:/msys64/mingw64/lib -lgrpc

d) finally rund devtools::install()

But I am getting a compilation error:

C:/msys64/mingw64/include/grpc/impl/codegen/port_platform.h:47:2: error: #error "Please compile grpc with _WIN32_WINNT of at least 0x600 (aka Windows Vista)"
 #error \
  ^

e) How do I set the compilation flag in the Makevars.win?

Full output.

> devtools::install()
Installing grpc
"C:/PROGRA~1/MICROS~3/ROPEN~1/R-35~1.1/bin/x64/R" --no-site-file --no-environ --no-save --no-restore --quiet CMD  \
  INSTALL "C:/Users/wewol/prog/grpc-1" --library="C:/Users/wewol/OneDrive/Documents/R/win-library/3.5"  \
  --install-tests 

* installing *source* package 'grpc' ...
** libs
c:/Rtools/mingw_64/bin/g++ -m64 -I"C:/PROGRA~1/MICROS~3/ROPEN~1/R-35~1.1/include" -DNDEBUG -IC:/msys64/mingw64/include -I"C:/Users/wewol/OneDrive/Documents/R/win-library/3.5/Rcpp/include"   -I"C:/swarm/workspace/External-R-3.5.1/vendor/extsoft/include"     -O2 -Wall  -mtune=core2 -c client.cpp -o client.o
In file included from C:/msys64/mingw64/include/grpc/support/port_platform.h:22:0,
                 from C:/msys64/mingw64/include/grpc/grpc.h:22,
                 from client.cpp:2:
C:/msys64/mingw64/include/grpc/impl/codegen/port_platform.h:47:2: error: #error "Please compile grpc with _WIN32_WINNT of at least 0x600 (aka Windows Vista)"
 #error \
  ^
client.cpp: In function 'Rcpp::RawVector fetch(Rcpp::CharacterVector, Rcpp::CharacterVector, Rcpp::RawVector)':
client.cpp:80:7: warning: unused variable 'was_cancelled' [-Wunused-variable]
   int was_cancelled = 2;
       ^
make: *** [C:/PROGRA~1/MICROS~3/ROPEN~1/R-35~1.1/etc/x64/Makeconf:215: client.o] Error 1
ERROR: compilation failed for package 'grpc'
* removing 'C:/Users/wewol/OneDrive/Documents/R/win-library/3.5/grpc'
In R CMD INSTALL
Error: Command failed (1)

nfultz · 2018-10-02T20:48:28Z

See also grpc/grpc#4707

wolski · 2018-10-03T06:38:52Z

Updated Makevars.win to:

PKG_CPPFLAGS=-D_WIN32_WINNT=0x600 -IC:/msys64/mingw64/include
PKG_LIBS=-LC:/msys64/mingw64/lib -lgrpc -lgpr

And it seems to build. There are now in the src folder :

$ ls *.o *.dll
client.o  grpc.dll*  onload.o  rcpp_hello.o  RcppExports.o  server.o

But now the installation process fails with :

Error: package or namespace load failed for 'grpc' in inDL(x, as.logical(local), as.logical(now), ...):
 unable to load shared object 'C:/Users/wewol/OneDrive/Documents/R/win-library/3.5/grpc/libs/x64/grpc.dll':
  LoadLibrary failure:  The specified module could not be found.

Full output:

> devtools::install()
Installing grpc
"C:/PROGRA~1/MICROS~3/ROPEN~1/R-35~1.1/bin/x64/R" --no-site-file --no-environ --no-save --no-restore --quiet CMD  \
  INSTALL "C:/Users/wewol/prog/grpc-1" --library="C:/Users/wewol/OneDrive/Documents/R/win-library/3.5"  \
  --install-tests 

* installing *source* package 'grpc' ...
** libs
make: Nothing to be done for 'all'.
installing to C:/Users/wewol/OneDrive/Documents/R/win-library/3.5/grpc/libs/x64
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error: package or namespace load failed for 'grpc' in inDL(x, as.logical(local), as.logical(now), ...):
 unable to load shared object 'C:/Users/wewol/OneDrive/Documents/R/win-library/3.5/grpc/libs/x64/grpc.dll':
  LoadLibrary failure:  The specified module could not be found.

Error: loading failed
Execution halted
ERROR: loading failed
* removing 'C:/Users/wewol/OneDrive/Documents/R/win-library/3.5/grpc'
In R CMD INSTALL
Error: Command failed (1)

nfultz · 2018-10-03T15:05:55Z

That's pretty mysterious; you might try asking on R-pkg-devel or the R-windows mailing lists.

Shehab7osny · 2020-04-06T11:39:51Z

@wolski @nfultz
Well, after some research, I came out with the following solution:

Download the package as a .zip file on your local storage.
Make a new Makevars.win file in the downloaded package and add the following to this file:
PKG_CPPFLAGS=-D_WIN32_WINNT=0x600 -IC:/msys64/mingw64/include
PKG_LIBS=-LC:/msys64/mingw64/lib -lgrpc -lgpr
Run the following command line in your Rstudio console:
install_local("**Path To Your Zip File**", INSTALL_opts = c("--no-multiarch", "--no-test-load"))
The package will be installed successfully, however you will need an extra more steps to avoid any future errors.
Close the Rstudio, and then move to your Packages Library and navigate to the grpc.dll file (Yours are located at C:/Users/wewol/OneDrive/Documents/R/win-library/3.5/grpc/libs/x64/).
Replace the existing dll file with the following grpc.dll file:
https://github.com/Shehab7osny/GSoc2020-GRPC-Tests/blob/master/Hard/grpc.dll
Open Rstudio one more time and load the package using library(grpc) and it will hopefully load.

If there are any additional problems it will be mainly in the grpc.dll file so you will need to generate it yourself as I did but I hope that it will work for you directly.

nfultz · 2020-04-07T15:33:56Z

Could you explain step 6 - where did that file come from?

Shehab7osny · 2020-04-08T16:36:42Z

Well, there are many cases that can lead to this error while loading the package on Rstudio (on Windows OS). I tried all of the possible ways to solve this problem but unfortunately none of them worked. However, according to the following issue, it was stated that the dll file should be generated on the system itself (depending of its specs). So, my approach was targeted to build the package files myself relying on my system specs (Windows 10 64-bit). I followed the following steps to build generate the .dll file:

Add a new Rcpp Package Project to my working area.
For the Implemented Rcpp Functions:
Add these files to the /src folder in the project
For the Source Files of the underlying C library:
Add these files to the /src folder in the project
For the Header Files of the underlying C library:
Add these files to the following directory C:/msys64/mingw64/include
Add a new Makevars.win file including the following:

CXX_STD = CXX11

PKGOBJECTS = [ List of Object Files ]
OBJECTS= $(PKGOBJECTS)

PKG_CPPFLAGS=-IC:/msys64/mingw64/include -D_WIN32_WINNT=0x600
PKG_LIBS= $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)

USERDIR	= 	C:/msys64/mingw64/lib
STATICLIB =	libgrpc.a
USERLIB	= 	$(USERDIR)/$(STATICLIB)

RM = 		rm -f

.PHONY: 	all clean

all: 		userlib $(SHLIB)

clean:
		${RM} $(OBJECTS) $(SHLIB)

$(USERLIB):	$(OBJECTS)

userlib:	$(STATICLIB)
		-mkdir -p $(USERDIR)
		-mv $(STATICLIB) $(USERLIB) 

$(STATICLIB):	$(OBJECTS)

This will generate a new static library libgrpc.a in the directory C:/msys64/mingw64/lib

Clean and Build the project.
Remove the Source Files of the underlying C library from the /src folder
Modify the Makevars.win file as follows:
PKG_CPPFLAGS=-D_WIN32_WINNT=0x600 -IC:/msys64/mingw64/include
PKG_LIBS=-LC:/msys64/mingw64/lib -lgrpc -lgpr
Clean and Build the project one more time and the grpc.dll file will be generated in the project directory

This was the only process that worked for me. I did intensive research and there are other solutions for this type of error but as I stated before none of them worked for me. However, the main two solutions that solved this problem for other users were as follows:

Check that grpc.dll (provided by any kind of different application) is not present on the PATH variables itself.
Details are included in the following link
Unblock grpc.dll files (as shown in the image)

nfultz · 2020-04-08T17:09:10Z

I see, quite complicated. The "unblock" thing in particular seems annoying.

I guess my thoughts are -

Can you try a make install for grpc C library on windows and see if it places the binary and header in the correct location?
How do other windows R packages handle this? I would expect the rprotobuf package to have very similar challenges and would generally recommend following Dirk's lead in this case.
Could these build steps be integrated in to a configure or configure.win script and / or Makefile?

I'm not an expert on windows packaging or autotools, but I would be surprised if we were the first ones to encounter DLL issues.

If rprotobuf doesn't have a solution, I'd recommend asking around at either the r-package-devel list or maybe the r-windows list.

Shehab7osny · 2020-04-08T18:08:13Z

You are definitely right! This approach is quite complicated. The good news is that we are not the first ones to encounter dynamic linking issues. It is actually quite common among Windows users trying to install various R packages. However, regarding your thoughts,

I will check this approach and let you know the results.
I tried many windows R packages available on CRAN, there were no issues at all while the installation process. They were actually installed and loaded pretty easily and straightforward. I definitely checked the Makevars.win file of those packages and they were similar to this one where the PKG_LIBS refers to a static lib (.a) file in the package directory. So, in my opinion, I think that there is something wrong in the grpc static library named libgrpc.dll.a which is built using pacman -S mingw-w64-x86_64-grpc. I think that there is another way to build this library file on windows and that what I'm searching about right now. I came to this assumption because when I built the static library myself using Rtools I didn't face this problem at all.
Actually right now I don't figure a way to do this but I will definitely try to.

I will try to follow Dirk's lead right now and see how exactly the rprotobuf package handles this DLL issue correctly.

nfultz added the windows label Nov 13, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Installing on windows #21

Installing on windows #21

wolski commented Oct 2, 2018

nfultz commented Oct 2, 2018

nfultz commented Oct 2, 2018

wolski commented Oct 2, 2018 •

edited

Loading

nfultz commented Oct 2, 2018

wolski commented Oct 3, 2018 •

edited

Loading

nfultz commented Oct 3, 2018

Shehab7osny commented Apr 6, 2020 •

edited

Loading

nfultz commented Apr 7, 2020

Shehab7osny commented Apr 8, 2020 •

edited

Loading

nfultz commented Apr 8, 2020

Shehab7osny commented Apr 8, 2020 •

edited

Loading

Installing on windows #21

Installing on windows #21

Comments

wolski commented Oct 2, 2018

nfultz commented Oct 2, 2018

nfultz commented Oct 2, 2018

wolski commented Oct 2, 2018 • edited Loading

nfultz commented Oct 2, 2018

wolski commented Oct 3, 2018 • edited Loading

nfultz commented Oct 3, 2018

Shehab7osny commented Apr 6, 2020 • edited Loading

nfultz commented Apr 7, 2020

Shehab7osny commented Apr 8, 2020 • edited Loading

nfultz commented Apr 8, 2020

Shehab7osny commented Apr 8, 2020 • edited Loading

wolski commented Oct 2, 2018 •

edited

Loading

wolski commented Oct 3, 2018 •

edited

Loading

Shehab7osny commented Apr 6, 2020 •

edited

Loading

Shehab7osny commented Apr 8, 2020 •

edited

Loading

Shehab7osny commented Apr 8, 2020 •

edited

Loading