New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

加速模型载入 #202

Open

shouldsee opened this issue Jul 18, 2024 · 2 comments

shouldsee commented Jul 18, 2024

RWKV6

目前模型载入的速度挺久的，需要载入ninja2 extension。开发的时候经常重新载入模型，有没有啥好办法加速？目前用的gradio应用基座，每次更改应用层逻辑的时候都会重载。

Owner

BlinkDL commented Jul 20, 2024

可以试试 https://github.com/Ai00-X/ai00_server

Author

shouldsee commented Jul 23, 2024

好的谢谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment