Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract and speed up device transfers #357

Merged
merged 4 commits into from
Oct 14, 2024
Merged

Extract and speed up device transfers #357

merged 4 commits into from
Oct 14, 2024

Conversation

frostedoyster
Copy link
Collaborator

@frostedoyster frostedoyster commented Oct 14, 2024

Moving systems to the device can be optimized as it requires moving many tensors sequentially (cell, positions, types, NLs, times the number of systems in a batch).

no compilation: 4.7 ms
script: 3.9 ms
torch.compile: 4.0 ms


📚 Documentation preview 📚: https://metatrain--357.org.readthedocs.build/en/357/

@frostedoyster
Copy link
Collaborator Author

@Luthaf do you know what's happening in the docs?

@Luthaf
Copy link
Member

Luthaf commented Oct 14, 2024

How does this speed it up? This is faster by using torchscript? Also the gains you show above are negligible, do you also have timings for a more realistic training set?


@Luthaf do you know what's happening in the docs?

This looks like trying to compile to torchscript the documentation classes, which only contain documentation. Maybe disabling TorchScript for the docs would help? There is an environment variable for this.

@frostedoyster
Copy link
Collaborator Author

The speed-up is nice, but I think the major advantage is to extract a function that would be essentially used everywhere

@frostedoyster
Copy link
Collaborator Author

We're already using the environment variable, right?

@Luthaf
Copy link
Member

Luthaf commented Oct 14, 2024

The docs are already using one env variable to switch between the TorchScript version of the classes and the documentation version of the classes. There is a separate env variable to disable TorchScript compilation altogether as well.

@frostedoyster frostedoyster merged commit 6ece48e into main Oct 14, 2024
12 checks passed
@frostedoyster frostedoyster deleted the transfers branch October 14, 2024 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants