Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update Llama2 tutorial for running with trtllm #73

Closed
wants to merge 14 commits into from
Closed

update Llama2 tutorial for running with trtllm #73

wants to merge 14 commits into from

Conversation

jbkyang-nvi
Copy link
Collaborator

@jbkyang-nvi jbkyang-nvi commented Dec 7, 2023 •
edited
Loading

No description provided.

pskiran1
pskiran1 requested changes Dec 7, 2023
View reviewed changes
Popular_Models_Guide/Llama2/trtllm_guide.md Outdated Show resolved Hide resolved
Popular_Models_Guide/Llama2/vllm_guide.md Outdated Show resolved Hide resolved
Popular_Models_Guide/Llama2/trtllm_guide.md Show resolved Hide resolved
fpetrini15
fpetrini15 reviewed Dec 8, 2023
View reviewed changes
Copy link
Collaborator

@fpetrini15 fpetrini15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks really good Katherine! Just a few minor comments.

Popular_Models_Guide/Llama2/README.md Outdated Show resolved Hide resolved
Popular_Models_Guide/Llama2/trtllm_guide.md Outdated Show resolved Hide resolved
Quick_Deploy/HuggingFaceTransformers/README.md Outdated Show resolved Hide resolved
Quick_Deploy/HuggingFaceTransformers/llama7b/config.pbtxt Outdated Show resolved Hide resolved
Popular_Models_Guide/Llama2/vllm_guide.md Outdated Show resolved Hide resolved
kaiyux
kaiyux reviewed Dec 13, 2023
View reviewed changes
Popular_Models_Guide/Llama2/trtllm_guide.md Outdated
```bash
mkdir -p /opt/tritonserver/model_repository
cp -r /tensorrtllm_backend/all_models/inflight_batcher_llm/* /opt/tritonserver/model_repository/.
rm -r /opt/tritonserver/model_repository/tensorrt_llm_bls
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to delete tensorrt_llm_bls

@jbkyang-nvi jbkyang-nvi mentioned this pull request Dec 14, 2023
Merged
pskiran1
pskiran1 previously approved these changes Dec 14, 2023
View reviewed changes
@jbkyang-nvi jbkyang-nvi changed the title update Llama2 tutorial for running with vLLM backend and general python based backends. update Llama2 tutorial for running with trtllm Dec 19, 2023
@prnake prnake mentioned this pull request Feb 15, 2024
Merged
@jbkyang-nvi
Copy link
Collaborator Author

addressed in #81

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pskiran1 pskiran1 pskiran1 left review comments

@nnshah1 nnshah1 Awaiting requested review from nnshah1

@fpetrini15 fpetrini15 Awaiting requested review from fpetrini15

@kaiyux kaiyux Awaiting requested review from kaiyux

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jbkyang-nvi @fpetrini15 @kaiyux @pskiran1