New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

请求官方出一个 4bit的量化版本，并附带模型的效果指标 #264

Open

qiangruoyu opened this issue Feb 5, 2025 · 1 comment

qiangruoyu commented Feb 5, 2025

希望能有一个4bit的官方量化版本。第三方量化都是没有详细的指标参数，即使使用了也不确信是否有严重掉点。
有了4bit的量化单个8卡的节点就能直接部署了。

Charmnut commented Feb 6, 2025

the same issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment