Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI @ HPC tips #33

Open
s-sajid-ali opened this issue Jan 30, 2025 · 3 comments
Open

AI @ HPC tips #33

s-sajid-ali opened this issue Jan 30, 2025 · 3 comments

Comments

@s-sajid-ali
Copy link
Collaborator

We need a rewrite of this whole section and rethink what's needed. I don't think running Shiny on GCP needs to be here.

Current Portal New Portal Status Notes
Large number of small files Move over to HPC storage
SQLite Move over to HPC storage guides
Joblib and Dask Needs porting, but I don't see why they need to be in AI @ HPC
LLM on HPC
@s-sajid-ali
Copy link
Collaborator Author

Add a page similar to https://docs.lxp.lu/howto/llama3-vllm/

Investigate and mention the use of https://github.com/VectorInstitute/vector-inference

@s-sajid-ali
Copy link
Collaborator Author

s-sajid-ali commented Feb 12, 2025

Integrate the guide on fine tuning LLMs on HPC as part of our docs. Perhaps fine tune a smaller model, like phi4 (https://huggingface.co/microsoft/phi-4).

And if we're feeling proactive, add a guide for LoRA fine tuning on HPC as well!

Fine-tuning-Llama2.pdf

@s-sajid-ali
Copy link
Collaborator Author

Outline added in #54

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant