Where is the ExT5 checkpoint? #998
-
|
Hi, I notice your work "EXT5: TOWARDS EXTREME MULTI-TASK SCALING FOR TRANSFER LEARNING" and you said "All of the modeling and training code used for EXT5 and its variants is already open-sourced as a part of the Mesh Tensorflow and T5 Libraries" in the section "REPRODUCABILITY STATEMENT". However, I cannot find them. May you tell me how to get them? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hey Steven, thanks for your interest in ExT5. The modeling and training code is open-sourced. See t5/models/gin/models/t5.1.1.base.gin (and large/xl/xxl) for the model architecture, and we used the default T5 training scripts to train it. However, I assume you are also asking about the model checkpoints and seqio implementations of the datasets. Unfortunately, because of the large number of datasets we use and the complexities of their licenses, it has been difficult for us to get legal approval to release the checkpoints or dataset processing code. While they may still be realeased in the distant future, there is no plan to release them in the immediate future. Hope you understand! |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your reply! |
Beta Was this translation helpful? Give feedback.
Hey Steven, thanks for your interest in ExT5.
The modeling and training code is open-sourced. See t5/models/gin/models/t5.1.1.base.gin (and large/xl/xxl) for the model architecture, and we used the default T5 training scripts to train it.
However, I assume you are also asking about the model checkpoints and seqio implementations of the datasets. Unfortunately, because of the large number of datasets we use and the complexities of their licenses, it has been difficult for us to get legal approval to release the checkpoints or dataset processing code. While they may still be realeased in the distant future, there is no plan to release them in the immediate future. Hope you understand!