Checkpoint for T5 trained on News data? #777
Unanswered
apoorvumang
asked this question in
Q&A
Replies: 1 comment 1 reply
-
Hi, no such checkpoint is available, but you can follow these instructions to train from scratch: |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi
Is it possible to get access to T5 pretrained model (any size) that was trained on just news data?
(Reason: I am working on KG completion on Wikidata using T5 and getting some convincing results. However, T5 pretraining using C4 means it has read Wikipedia which can be a source of test leakage. I am looking for any pretrained T5 that does not have wikipedia in its pretraining dataset.)
It would be very helpful for my research.
Also, if it's not possible, could you suggest how I should go about training it myself?
Thanks
Beta Was this translation helpful? Give feedback.
All reactions