You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello expert:
I used my own data to fine tune the Moirai large model according to the fine-tuning process and code in the official README file, and found that the effect after fine-tuning was actually worse than without fine-tuning, which is strange.
Among them, because I set the context length to 96 and the prediction length to 4 during prediction, I set the parameters in the cli/conf/finetune/val_data/data.yaml file as shown in the figure. The learning rate in moirai-1.0-R-small.yaml was set to 1e-7, and other places such as hyperparameters were not changed. I tried fine-tuning all layers of Moirai and some layers of the output layer separately, but the results were even worse.
Do you know where my mistake occurred? Do I need to make specific changes to the hyperparameter settings or other content during the fine-tuning process?
Looking forward to your reply, thank you very much!
The text was updated successfully, but these errors were encountered:
Hello expert:
I used my own data to fine tune the Moirai large model according to the fine-tuning process and code in the official README file, and found that the effect after fine-tuning was actually worse than without fine-tuning, which is strange.
Among them, because I set the context length to 96 and the prediction length to 4 during prediction, I set the parameters in the cli/conf/finetune/val_data/data.yaml file as shown in the figure. The learning rate in moirai-1.0-R-small.yaml was set to 1e-7, and other places such as hyperparameters were not changed. I tried fine-tuning all layers of Moirai and some layers of the output layer separately, but the results were even worse.
Do you know where my mistake occurred? Do I need to make specific changes to the hyperparameter settings or other content during the fine-tuning process?
Looking forward to your reply, thank you very much!
The text was updated successfully, but these errors were encountered: