Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DLL load failed for CUDA on Python 3.8 for Windows #2039

Open
cashlo opened this issue Mar 28, 2020 · 27 comments
Open

DLL load failed for CUDA on Python 3.8 for Windows #2039

cashlo opened this issue Mar 28, 2020 · 27 comments

Comments

@cashlo
Copy link

cashlo commented Mar 28, 2020

Expected Behavior

>>> import dlib
>>> :)

Current Behavior

>>> import dlib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing dlib: The specified module could not be found.
>>> :(

Steps to Reproduce

  1. Install CUDA
  2. Install cuDNN
  3. git clone https://github.com/davisking/dlib.git
  4. python setup.py install
  5. python -c"import dlib"

Because of A change in Python 3.8 for Windows, Python no longer searches for DLLs in directories listed in the PATH environment variable and fails to find curand64_10.dll, this is fixed after running os.add_dll_directory(os.path.join(os.environ['CUDA_PATH'], 'bin'))

  • Version: 19.19.99
  • Where did you get dlib: this github repo
  • Platform: Python 3.8 64-bit for Windows 10
@davisking
Copy link
Owner

Urg, windows dynamic loading is such a mess :(

@davisking
Copy link
Owner

I'm marking this help wanted because I'm not feeling super motivated to figure out how to fix this right now. So if someone wants to make this work I would much appreciate it.

It seems like the solution is to make CMake/setup.py generate a init.py that includes the appropriate os.add_dll_directory() call and then imports everything from the compiled dlib library users currently import.

@cashlo
Copy link
Author

cashlo commented Mar 28, 2020

Let me see if I can do that in the next few days, I have everything needed to reproduce it anyway.

@Mouradost
Copy link

Expected Behavior

>>> import dlib
>>> :)

Current Behavior

>>> import dlib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing dlib: The specified module could not be found.
>>> :(

Steps to Reproduce

  1. Install CUDA
  2. Install cuDNN
  3. git clone https://github.com/davisking/dlib.git
  4. python setup.py install
  5. python -c"import dlib"

Because of A change in Python 3.8 for Windows, Python no longer searches for DLLs in directories listed in the PATH environment variable and fails to find curand64_10.dll, this is fixed after running os.add_dll_directory(os.path.join(os.environ['CUDA_PATH'], 'bin'))

  • Version: 19.19.99
  • Where did you get dlib: this github repo
  • Platform: Python 3.8 64-bit for Windows 10

I followed the same steps but I'm still getting the error of DLL load failed while importing dlib: The specified module could not be found

@cashlo
Copy link
Author

cashlo commented May 28, 2020

Well, clearly I failed to work on this. This seems to be a much bigger change than I expected.
@Mouradost do you have the dlls in your CUDA_PATH?

@Mouradost
Copy link

@cashlo Yes I do since I'm using Tensorflow with the GPU here is my CUDA_PATHs

image

@issaiass
Copy link

issaiass commented Jun 7, 2020

For some reason:

Test1:
1 - Open a terminal
2 - Run the python command
3 - Impor dlib

Result: Causes DLL error

Test2:
1 - Open Terminal
2 - Run the python command
3 - import cv2
4 - import dlib

Result: No error, also, i checked the version! 💯

image

@davisking
Copy link
Owner

I just pushed a change to how dlib is packaged that should fix this. So pull the latest dlib from github and try installing that version.

@Mouradost
Copy link

For some reason:

Test1:
1 - Open a terminal
2 - Run the python command
3 - Impor dlib

Result: Causes DLL error

Test2:
1 - Open Terminal
2 - Run the python command
3 - import cv2
4 - import dlib

Result: No error, also, i checked the version! 💯

image

I just pushed a change to how dlib is packaged that should fix this. So pull the latest dlib from github and try installing that version.

still getting DLL errors
image

@davisking
Copy link
Owner

What's in your CUDA_PATH environment variable?

@Mouradost
Copy link

What's in your CUDA_PATH environment variable?

image

@davisking
Copy link
Owner

@Mouradost That doesn't look like the contents of the CUDA_PATH variable, but PATH or something else. I say this because it would be very surprising if all those c:\windows paths were in CUDA_PATH.

@Mouradost
Copy link

@Mouradost That doesn't look like the contents of the CUDA_PATH variable, but PATH or something else. I say this because it would be very surprising if all those c:\windows paths were in CUDA_PATH.

image
my bad sorry

@davisking
Copy link
Owner

So whats in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin?

@Mouradost
Copy link

Mouradost commented Jun 14, 2020

So whats in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin?

These are the files in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin
image

@davisking
Copy link
Owner

davisking commented Jun 16, 2020 via email

@Mouradost
Copy link

I don't see cudnn's .dll file in that list. You need to install cudnn. Which is curious, since cmake must have found a copy of cudnn somewhere on your computer based on other things you said.

On Sun, Jun 14, 2020 at 4:08 AM mourad lablack @.***> wrote: So whats in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin? These are the files in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin [image: image] https://user-images.githubusercontent.com/17816194/84588343-2be56e00-ae59-11ea-9146-d744a4594f25.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2039 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPYFRZH72KFTJVMOR5QMLTRWSAQXANCNFSM4LVOATAQ .

I have it under a different folder on c:\tools\cuda\bin should I add a copy to the C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin ?

@Mouradost
Copy link

Mouradost commented Jun 17, 2020

it's working awesome thank you @davisking
I wonder could you guys give away a possibility to specify the path for cudnn if it's not moved to the cuda main folder? or directly grab it from the windows paths? like tensorflow do

@davisking
Copy link
Owner

davisking commented Jun 17, 2020

Where was it? Oh you noted in your other comment. Really we should make the install process record the location of the copy it found rather than expecting the user to do something special.

@davisking
Copy link
Owner

davisking commented Jun 18, 2020

@Mouradost I just pushed a change to github that should make this work without needing to assume anything about CUDA_PATH. So try the latest dlib on github.

@Mouradost
Copy link

@Mouradost I just pushed a change to github that should make this work without needing to assume anything about CUDA_PATH. So try the latest dlib on github.

Sure I will give you my feedback thank you for your consideration and your help

@Mouradost
Copy link

Mouradost commented Jun 18, 2020

Where was it? Oh you noted in your other comment. Really we should make the install process record the location of the copy it found rather than expecting the user to do something special.

I had the habit to create a separate folder in the C:\ called tools and I put my cudnn files there which make it easy to find and change only the cudnn files if I want to use different version on cudnn without touching cuda that's why I thought it will be better to have your library allowing for additional paths.

@ElPrg
Copy link

ElPrg commented Sep 21, 2020

Expected Behavior

>>> import dlib
>>> :)

Current Behavior

>>> import dlib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing dlib: The specified module could not be found.
>>> :(

Steps to Reproduce

1. Install CUDA

2. Install cuDNN

3. `git clone https://github.com/davisking/dlib.git`

4. `python setup.py install`

5. `python -c"import dlib"`

Because of A change in Python 3.8 for Windows, Python no longer searches for DLLs in directories listed in the PATH environment variable and fails to find curand64_10.dll, this is fixed after running os.add_dll_directory(os.path.join(os.environ['CUDA_PATH'], 'bin'))

* **Version**: 19.19.99

* **Where did you get dlib**: this github repo

* **Platform**: Python 3.8 64-bit for Windows 10

I had a situation that dlib crashed when I did import it, in general I did the following. I changed setup.py and set it to compile Debug, but with Release it didn't work on Win 7 x 4 Py 3.7.

link on setup_for Debug compile
https://drive.google.com/file/d/1Wfy9ou3yYce4wqywY22jPxC9IkN6_Dch

@VasylKolomiiets
Copy link

VasylKolomiiets commented Apr 3, 2021

I instal version Dlib 19.7.0. on python 3,6.13

pip install https://pypi.python.org/packages/da/06/bd3e241c4eb0a662914b3b4875fc52dd176a9db0d4a2c915ac2ad8800e9e/dlib-19.7.0-cp36-cp36m-win_amd64.whl#md5=b7330a5b2d46420343fbed5df69e6a3f
all works... but may be any advices about new versions?

@VasylKolomiiets
Copy link

VasylKolomiiets commented Apr 3, 2021

I work in:

Windows10
CUDA10.1
Anaconda
Python 3.8

Steps to reproduce

  1. Install CUDA
  2. Install cuDNN
  3. Install Anaconda
  4. create env
  5. in terminal write conda install dlib -c conda-forge
    Environment (conda list):
> (dlib) C:\Users\Vasil>conda list
> # packages in environment at D:\Games\conda\envs\dlib:
> #
> # Name                    Version                   Build  Channel
> ca-certificates           2020.12.5            h5b45459_0    conda-forge
> certifi                   2020.12.5        py38haa244fe_1    conda-forge
> dlib                      19.21.1          py38h082dfc2_0    conda-forge
> intel-openmp              2020.3             h57928b3_311    conda-forge
> jpeg                      9d                   h8ffe710_0    conda-forge
> libblas                   3.9.0                     8_mkl    conda-forge
> libcblas                  3.9.0                     8_mkl    conda-forge
> liblapack                 3.9.0                     8_mkl    conda-forge
> libpng                    1.6.37               h1d00b33_2    conda-forge
> mkl                       2020.4             hb70f87d_311    conda-forge
> numpy                     1.20.1           py38h0cc643e_0    conda-forge
> openssl                   1.1.1k               h8ffe710_0    conda-forge
> pip                       21.0.1             pyhd8ed1ab_0    conda-forge
> python                    3.8.8           h7840368_0_cpython    conda-forge
> python_abi                3.8                      1_cp38    conda-forge
> setuptools                49.6.0           py38haa244fe_3    conda-forge
> sqlite                    3.35.3               h8ffe710_0    conda-forge
> vc                        14.2                 hb210afc_4    conda-forge
> vs2015_runtime            14.28.29325          h5e1d092_4    conda-forge
> wheel                     0.36.2             pyhd3deb0d_0    conda-forge
> wincertstore              0.2             py38haa244fe_1006    conda-forge
> zlib                      1.2.11            h62dcd97_1010    conda-forge
> 
> (dlib) C:\Users\Vasil>

Details about conda and system ( conda info ):

>     active environment : dlib
>     active env location : D:\Games\conda\envs\dlib
>             shell level : 1
>        user config file : C:\Users\Vasil\.condarc
>  populated config files : C:\Users\Vasil\.condarc
>           conda version : 4.9.2
>     conda-build version : 3.20.5
>          python version : 3.8.5.final.0
>        virtual packages : __cuda=11.2=0
>                           __win=0=0
>                           __archspec=1=x86_64
>        base environment : D:\Games\conda  (writable)
>            channel URLs : https://conda.anaconda.org/conda-forge/win-64
>                           https://conda.anaconda.org/conda-forge/noarch
>                           https://repo.anaconda.com/pkgs/main/win-64
>                           https://repo.anaconda.com/pkgs/main/noarch
>                           https://repo.anaconda.com/pkgs/r/win-64
>                           https://repo.anaconda.com/pkgs/r/noarch
>                           https://repo.anaconda.com/pkgs/msys2/win-64
>                           https://repo.anaconda.com/pkgs/msys2/noarch
>           package cache : D:\Games\conda\pkgs
>                           C:\Users\Vasil\.conda\pkgs
>                           C:\Users\Vasil\AppData\Local\conda\conda\pkgs
>        envs directories : D:\Games\conda\envs
>                           C:\Users\Vasil\.conda\envs
>                           C:\Users\Vasil\AppData\Local\conda\conda\envs
>                platform : win-64
>              user-agent : conda/4.9.2 requests/2.24.0 CPython/3.8.5 Windows/10 Windows/10.0.19041
>           administrator : False
>              netrc file : None
>            offline mode : False

What seams strange - that line virtual packages : __cuda=11.2=0 when I have CUDA10.1

All I did - just install dlib in new env and try to do import:

(dlib) C:\Users\Vasil>conda install dlib
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: D:\Games\conda\envs\dlib

  added / updated specs:
    - dlib


The following NEW packages will be INSTALLED:

  dlib               conda-forge/win-64::dlib-19.21.1-py38h082dfc2_0
  intel-openmp       conda-forge/win-64::intel-openmp-2020.3-h57928b3_311
  jpeg               conda-forge/win-64::jpeg-9d-h8ffe710_0
  libblas            conda-forge/win-64::libblas-3.9.0-8_mkl
  libcblas           conda-forge/win-64::libcblas-3.9.0-8_mkl
  liblapack          conda-forge/win-64::liblapack-3.9.0-8_mkl
  libpng             conda-forge/win-64::libpng-1.6.37-h1d00b33_2
  mkl                conda-forge/win-64::mkl-2020.4-hb70f87d_311
  numpy              conda-forge/win-64::numpy-1.20.1-py38h0cc643e_0
  zlib               conda-forge/win-64::zlib-1.2.11-h62dcd97_1010


Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

(dlib) C:\Users\Vasil>python
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 15:50:08) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import dlib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  
  File "D:\Games\conda\envs\dlib\lib\site-packages\dlib\__init__.py", line 19, in <module>
    from _dlib_pybind11 import *
ImportError: DLL load failed while importing _dlib_pybind11: A dynamic link library (DLL) initialization routine failed.

here is my windows PATH

CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1 CUDA_PATH_V10_1=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1 ...      
PATH=D:\Games\conda\envs\dlib;D:\Games\conda\envs\dlib\Library\mingw-w64\bin;
D:\Games\conda\envs\dlib\Library\usr\bin;D:\Games\conda\envs\dlib\Library\bin;
D:\Games\conda\envs\dlib\Scripts;
D:\Games\conda\envs\dlib\bin;
D:\Games\conda\condabin;
D:\Games\conda;
D:\Games\conda\Library\mingw-w64\bin;
D:\Games\conda\Library\usr\bin;
D:\Games\conda\Library\bin;
D:\Games\conda\Scripts;
C:\Program Files (x86)\Common Files\Oracle\Java\javapath;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\libnvvp;
C:\Windows\system32;C:\Windows;
C:\Windows\System32\Wbem;
C:\Windows\System32\WindowsPowerShell\v1.0;
C:\Windows\System32\OpenSSH;
C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;
C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.3.0;
C:\WINDOWS\system32;
C:\WINDOWS;
C:\WINDOWS\System32\Wbem;
C:\WINDOWS\System32\WindowsPowerShell\v1.0;
C:\WINDOWS\System32\OpenSSH;
C:\Program Files\PuTTY;
D:\Program Files\Git\cmd;
C:\Program Files\nodejs;C:\Program Files\dotnet;
C:\Users\Vasil\AppData\Local\Microsoft\WindowsApps;
C:\Users\Vasil\AppData\Local\Programs\Microsoft VS Code\bin;
C:\Users\Vasil\AppData\Local\GitHubDesktop\bin;
C:\Users\Vasil\AppData\Local\Microsoft\WindowsApps;
d:\Program Files\heroku\bin;C:\Users\Vasil\AppData\Roaming\npm;
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin;.

and cudnn64_7.dll is in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin

@hrkrx
Copy link

hrkrx commented Apr 30, 2021

I am facing the same error but on Python 3.9, I compiled it and everything seemed okay. But when i import dlib the before mentioned DLL error is displayed.

EDIT: I got Cuda to run in WSL 2, so I'll probably stick with it even if windows gets fixed. I'd suggest dropping windows all together once WSL 2 is outside of the Windows insider program

EDIT: after installing the expirmental CUDA for WSL driver it also worked on windows, but don't ask me why

@mtsnrtkhr
Copy link

I had same issue when I compile dlib with CUDA enabled.
In my case, the reason is the path of cuDNN.

"C:\Program Files\NVIDIA\CUDNN\v8.9.7.29\lib\cudnn.lib"

"import dlib" failed with that error.

When I move the filed under lib to "x64" as originally from cuDNN,zip

"C:\Program Files\NVIDIA\CUDNN\v8.9.7.29\lib\x64\cudnn.lib"

the problem solved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants