v0.0.8: Independent evaluation and k_values supported; fixed bugs & use load_sbert in mnrl and evaluation
Independent evaluation and k_values supported
One can now run the gpl.toolkit.evaluation
directly. Previously, it was only possible as part of the whole gpl.train
workflow. Please check this example for more details.
And we have also added argument k_values
in gpl.toolkit.evaluation.evaluate
. This is for specifying the K values in "nDCG@K", "recall@K", etc.
Fixed bugs & use load_sbert in mnrl and evaluation
Now almost all methods that require a separation token has this argument called sep
(previously it was fixed as a blank token " "
). Two exceptions are mnrl
(a loss function in SBERT repo, also the default training loss for the QGen method) and qgen
, since they are from the BeIR repo (we will update the BeIR repo in the future if possible).