New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat-26/AMP를 적용하여 실험시간 줄이기 #11

Open

2 tasks done

kimmaru opened this issue Nov 15, 2024 · 1 comment

Open

2 tasks done

feat-26/AMP를 적용하여 실험시간 줄이기 #11

kimmaru opened this issue Nov 15, 2024 · 1 comment

Assignees

Labels

Contributor

kimmaru commented Nov 15, 2024 •

edited

Loading

Background

실험에 걸리는 시간을 단축시키기 위해 AMP(Automatic Mixed Precision)을 적용하고자함

todo

train시에 amp를 적용하여 fp16으로 weight를 학습시키기
적용 전 후 실험 시간 비교

reference

노션 TODO ID-26
https://developer.nvidia.com/automatic-mixed-precision

kimmaru added the enhancement label

kimmaru self-assigned this

Contributor Author

kimmaru commented Nov 15, 2024 •

edited

Loading

적용 전 : 5epoch당 9분 ~ 9분30초 소요
적용 후 : 5epoch당 9분 ~ 9분30초 소요

동일하게 소요되고 있는 것을 확인.

가설 :

mixed precision이 제대로 적용되지 않고 있을 경우
모델의 크기가 충분히 크지 않아 효과가 눈에 띄지 않는 경우

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment