Leaderboard Gradio or Dash? #1277
Replies: 2 comments
-
I'm just gonna tag you, so that you can chime in @KennethEnevoldsen @orionw @Muennighoff @vaibhavad @imenelydiaker @isaac-chung @tomaarsen |
Beta Was this translation helpful? Give feedback.
-
Thanks @x-tabdeveloping for the great work! I think one highlight of the current leaderboard is that anyone can come in and edit it/make PRs. I am a bit hesitant to move off of the simple supported libraries like Gradio and Streamlit. OTOH, I'm certain you would create a very slick UI in Dash though and we could probably pick it up -- plus how much do we really change the leaderboard beyond adding models and/or new tabs? So I would say I probably lean against switching, but could be convinced if the UI is that much better for Dash/some other framework. But I haven't been working with the leaderboard for long so I defer to others. |
Beta Was this translation helpful? Give feedback.
-
Hey guys! I have been working on the new leaderboard for MMTEB for the last bit, and I have a couple of considerations that I want to put out here so that we can take an informed decision together, as I expect that some of these things might affect some of you just as much as me.
I started working on the leaderboard in the Gradio framework, which is thee one that was used for the original leaderboard too.
I must say, that I find it quite limiting and I wanted to ask you if it would be okay with you to base the new leaderboard on Dash instead.
Here are a couple of advantages of switching to Dash:
On the other hand there are also quite a few disadvantages to switching to Dash, namely:
I'm curious to hear your opinions about this. I don't have terribly strong feelings about the issue, and I'm sure that I could still push out a decent leaderboard in Gradio, but I feel like it would make my job easier if I was allowed to use Dash.
Beta Was this translation helpful? Give feedback.
All reactions