-
Notifications
You must be signed in to change notification settings - Fork 287
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CUDNN 9 support #1780
Comments
Considering whether upgrading to PyTorch >=2.4.0 is necessary at this time to avoid impacting users on cuDNN 8. However, it might be better to follow PyTorch and upgrade to cuDNN 9. |
Just a heads up, I was able to compile it succesfully against CUDA 12.4 and CUDNN9 without any code changes. I used the pytorch/pytorch:2.4.0-cuda12.4-cudnn9-devel docker image and updated the Dockerfile somewhat, then copied /opt/ctranslate2 to my runtime and installed the wheel there and it works without an issue (I needed faster-wisper) |
Hey, would you mind sharing your Dockerfile and any additional relevant commands you've used? I'm trying to switch over project faster-whisper-server over to latest CUDA with CUDNN9. Thanks! |
Build it with If you have problems, I've pushed the image to the docker container registry The dockerfile is mostly the same, I got a circular dependency with the multistage build for some reason and I couldn't spot the issue quickly so I did away with that. I had to install the cudnn9 dev kit with |
currently trying to use whisperX (which uses faster whisper, which uses CTranslate2), and am getting the following error
Could not locate cudnn_ops_infer64_8.dll. Please make sure it is in your library path!
which from what I can tell is due to having CUDNN version > 9. This is an issue because it looks like pytorch >= 2.4.0 are compiled with CUDNN > 9.
See discussion here
SYSTRAN/faster-whisper#958
Workaround right now is just to use a pytorch version < 2.4
The text was updated successfully, but these errors were encountered: