Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash in CPU-only environments by skipping torch.cuda.synchronize() #193

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rayfiyo
Copy link

@rayfiyo rayfiyo commented Jan 25, 2025

Checklist

  • Read CONTRIBUTING.md, and accept the CLA by including the provided snippet. We will not accept PR without this.
  • Run pre-commit hook.
  • If you changed Rust code, run cargo check, cargo clippy, cargo test.

PR Description

Overview

This PR fixes a crash that occurs when running on CPU-only environments
(e.g., --device cpu or no NVIDIA driver).
Previously, torch.cuda.synchronize() was always called in warmup(),
leading to an AssertionError on CPU builds.

Changes

  • In server.pywarmup(), the torch.cuda.synchronize() call is now
    wrapped with a condition checking if --device contains "cuda".

Testing

  • Confirmed no crash on a CPU-only setup (pip install torch --index-url https://download.pytorch.org/whl/cpu).
  • GPU usage remains unaffected.

Actual command

I tested on Docker's debian:12.5-slim

pip install --upgrade pip --break-system-packages
pip install torch torchvision torchaudio --break-system-packages --index-url https://download.pytorch.org/whl/cpu
pip install moshi --break-system-packages
python3 -m moshi.server --device cpu

CLA

I, rayfiyo, confirm that I have read and understood the terms of the CLA of Kyutai-labs,
as outlined in the repository's CONTRIBUTING.md, and I agree to be bound by these terms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant