We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hello,
Model https://huggingface.co/databricks/dbrx-instruct is not working with Flash attn on ROCM, working on NVIDIA 100
The current model is not working with AMD MI250 with flash attention:
Concretly take a node of MI250 : load with attn_implementation="flash_attention_2"
See: https://huggingface.co/databricks/dbrx-instruct
ADASTRA
AMD Instinct MI250X, AMD Instinct MI250
ROCm 6.0.0
No response
https://huggingface.co/databricks/dbrx-instruct/discussions/13
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Problem Description
Hello,
Model https://huggingface.co/databricks/dbrx-instruct is not working with Flash attn on ROCM, working on NVIDIA 100
The current model is not working with AMD MI250 with flash attention:
Concretly take a node of MI250 :
load with attn_implementation="flash_attention_2"
See: https://huggingface.co/databricks/dbrx-instruct
Operating System
ADASTRA
CPU
ADASTRA
GPU
AMD Instinct MI250X, AMD Instinct MI250
ROCm Version
ROCm 6.0.0
ROCm Component
No response
Steps to Reproduce
https://huggingface.co/databricks/dbrx-instruct/discussions/13
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response
The text was updated successfully, but these errors were encountered: