Implement vLLM FSDP LoRA hot-swapping integration #10
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added support for non-fsdp models.
trainer: replaced clip_grad_norm_ with nn.utils.clip_grad_norm_ for lora compatibility.
Set model path to local copy of llama-2-7b in example config.
…is method no longer wraps load_model_and_tokenizer) test_modelling: revised base model fixture scope since torch FSDP wrap is in-place. launch_benchmark: added confirmation before launching.
…enchmarking * added changes to implement low cpu mem usage feature * implemented new ruff linting changes and ran a fix across files
…s/config.md accordingly.
…ng configs and documentations.
Still need to move barrier logic into _VLLMCallbackWrapper.
Cleanup is required.
Cleanup is required.
Cleanup is required.
…mize changes required in llama_example.py.
adil-a
reviewed
May 28, 2024
docs/config.md
Outdated
|
|
||
| ### Sampling during Training | ||
|
|
||
| To disable sampling during training, delete the entire "sampling" section. |
There was a problem hiding this comment.
Are we "deleting" the section or just commenting out?
There was a problem hiding this comment.
"Comment out" might be sufficient, as it allows the user to easily re-enabled the sampling engine as needed.
adil-a
reviewed
May 28, 2024
configs/config_gemma.yaml
Outdated
There was a problem hiding this comment.
Is this file required to be a part of the main codebase?
There was a problem hiding this comment.
That config file has been included by mistake. I will delete that from version control.
adil-a
reviewed
May 28, 2024
docs/sampling.md
Outdated
adil-a
reviewed
May 28, 2024
configs/config_gemma.yaml
Outdated
| wandb_config: | ||
| project: vector-lm-verify | ||
| name: benchmark-lora | ||
| # tags: ["20240418-1a-preemption"] |
adil-a
reviewed
May 28, 2024
examples/__init__.py
Outdated
There was a problem hiding this comment.
Don't need an init file in examples. It's not part of the package installation.
There was a problem hiding this comment.
Sounds good. I have also added some verification logic to ensure that users are invoking the wrapper correctly.
adil-a
reviewed
May 28, 2024
…d importing vLLM when not required. Ruff formatting fixes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request enables vLLM to run in parallel with VectorLM on the same set of GPUs. Additionally, this pull request includes an example of LoRA adapter hot-swapping for tracking the behavior of the model during the training process.