Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

P-tuning中模型参数冷冻的问题 #42

Open
YCC-324 opened this issue May 12, 2023 · 2 comments
Open

P-tuning中模型参数冷冻的问题 #42

YCC-324 opened this issue May 12, 2023 · 2 comments

Comments

@YCC-324
Copy link

YCC-324 commented May 12, 2023 •
edited
Loading

您好，我理解的p-tuning的原理是冷冻后面大语言模型，只调整前面的prompt embedding模型。但是在您代码的实现中(https://github.com/THUDM/P-tuning/blob/main/PT-Fewshot/pet/wrapper.py 中optimizer部分) 同时对后面大语言模型的参数进行了微调，想问下这部分是我理解错了吗
@Deerkangkang
Copy link

我理解意义上也是需要对整体模型进行微调

@qiuwenbogdut
Copy link

您好，我理解的p-tuning的原理是冷冻后面大语言模型，只调整前面的prompt embedding模型。但是在您代码的实现中(https://github.com/THUDM/P-tuning/blob/main/PT-Fewshot/pet/wrapper.py 中optimizer部分) 同时对后面大语言模型的参数进行了微调，想问下这部分是我理解错了吗

可以参考一下之前的一个issue #4
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@qiuwenbogdut @YCC-324 @Deerkangkang