Registring models and results

[imenelydiaker]

Some points to discuss: (@KennethEnevoldsen @MartinBernstorff @Muennighoff feel free to edit my post and add details)

• Registering models: have something like ModelMetaData
• Move results folder to the current reposiroty (see how github submodules work, in my mind it's just a pointer to another github repository and doesn't store anything but it may not be the case 🤔)

[KennethEnevoldsen]

Move results folder to the current reposiroty (see how github submodules work, in my mind it's just a pointer to another github repository and doesn't store anything but it may not be the case 🤔)

Alternatively we could also move it all to the GitHub (we don't need to have the results on HF

[Myahr208]

In

[KennethEnevoldsen]

@tomaarsen just saw some of the work you have already been doing on the MTEB benchmark. I believe you have essentially already implemented a ModelMeta. The reason why we want to add it to GitHub is to allow for the reproducibility of model results (e.g. which prompts were used). Thus a submission of the model's results will also include a script on how it was run.

Similarly, as you already work with the serving of the benchmark I want to make sure to also get your opinion on moving the results to GitHub.