-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GPU tests failing #291
Comments
this is the last one that doesn't nan: and the first one which nans but this ci run was just after change to README (and before enabling openmp). Smells like something in the CUDA toolchain?! |
There are quite a few versions changes so not sure what the culprit is. |
The successful one uses Installed CUDA_Driver_jll ── v0.7.0+1
Installed CUDA_Runtime_jll ─ v0.11.1+0
Installed SCS_GPU_jll ────── v3.2.4+0 The failing one does Installed SCS_GPU_jll ────── v3.2.4+0
Installed CUDA_Driver_jll ── v0.8.0+0
Installed CUDA_Runtime_jll ─ v0.12.0+1 So this seems to be a problem with cuda-12? @maleadt (sorry if you get too many pings) |
Upgrading CUDA_Runtime_jll only updates the underlying CUDA toolkit. Maybe your package is incompatible with the CUDA toolkit v12.4 as introduced by Runtime_jll 0.12, or needs a rebuild. |
@maleadt It seems that the newest scs was already built against CUDA toolkit 12.4/5: @bodono did you test scs with CUDA-12? some examples here run just fine (so I think we're interacting with the library correctly), but some end with bunch of nans. |
Unfortunately if CUDA 12 is newish then it's likely that I have never tested with it, since I no longer have access to a GPU machine. The github action I have for gpus only compiles it. |
cc @kalmarek
Some examples have
nan
:The text was updated successfully, but these errors were encountered: