Fix crash in CPU-only environments by skipping torch.cuda.synchronize() #193
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
cargo check
,cargo clippy
,cargo test
.PR Description
Overview
This PR fixes a crash that occurs when running on CPU-only environments
(e.g.,
--device cpu
or no NVIDIA driver).Previously,
torch.cuda.synchronize()
was always called inwarmup()
,leading to an
AssertionError
on CPU builds.Changes
server.py
→warmup()
, thetorch.cuda.synchronize()
call is nowwrapped with a condition checking if
--device
contains"cuda"
.Testing
pip install torch --index-url https://download.pytorch.org/whl/cpu
).Actual command
I tested on Docker's debian:12.5-slim
CLA
I, rayfiyo, confirm that I have read and understood the terms of the CLA of Kyutai-labs,
as outlined in the repository's CONTRIBUTING.md, and I agree to be bound by these terms.