Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README: ensure modeling code is patched before model instantiation #170

Merged
merged 3 commits into from
Aug 30, 2024
Merged

README: ensure modeling code is patched before model instantiation #170

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
56defb9
README: ensure modeling code is patched before model instantiation
tmm1 Aug 29, 2024
f63ac99
Merge branch 'main' into fix-examples
shimizust Aug 30, 2024
743d0b9
Merge branch 'main' into fix-examples
ByronHsu Aug 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,19 +120,20 @@ Using the [patching APIs](#patching), you can swap Hugging Face models with opti
import transformers
from liger_kernel.transformers import apply_liger_kernel_to_llama

model = transformers.AutoModelForCausalLM("path/to/llama/model")

# Adding this line automatically monkey-patches the model with the optimized Liger kernels
# 1a. Adding this line automatically monkey-patches the model with the optimized Liger kernels
apply_liger_kernel_to_llama()

# You could alternatively specify exactly which kernels are applied
# 1b. You could alternatively specify exactly which kernels are applied
apply_liger_kernel_to_llama(
rope=True,
swiglu=True,
cross_entropy=True,
fused_linear_cross_entropy=False,
rms_norm=False
)

# 2. Instantiate patched model
model = transformers.AutoModelForCausalLM("path/to/llama/model")
```

### 3. Compose Your Own Model
Expand Down
Loading