Skip to content

Issues: embeddings-benchmark/arena

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

24 Open 7 Closed
24 Open 7 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Make the arena a competitive arXiv/Wikipedia/StackExchange search engine
#44 opened Sep 16, 2024 by Muennighoff
4 tasks
1
allow downloading query embedding
#42 opened Sep 12, 2024 by Muennighoff
Breakdown Across Corpus
#41 opened Aug 23, 2024 by zanussbaum
Diff between retrieved documents
#40 opened Aug 15, 2024 by Muennighoff
Top1 vs Top2 of same model
#39 opened Aug 15, 2024 by Muennighoff
Do more to prevent ties
#36 opened Aug 12, 2024 by KennethEnevoldsen
2
Normalization during retrieval scores computation
#34 opened Aug 6, 2024 by violenil
2
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'mteb/index_stackexchange_### model a: bm25'.
#33 opened Aug 5, 2024 by Muennighoff
3
Support for Retrieval-only models (ColBERT?)
#32 opened Aug 5, 2024 by bclavie
7
Check "both are bad" / "tie" votes & consider displaying top-K
#31 opened Aug 1, 2024 by Muennighoff
2
Connect leaderboard with crowdsourcing platform
#29 opened Jul 31, 2024 by Muennighoff
Skip responses when passages are the same?
#28 opened Jul 31, 2024 by orionw
1
Use the CI in determining ranking numbers
#27 opened Jul 31, 2024 by orionw
Turn MTEB-Arena-logs into HF dataset?
#25 opened Jul 30, 2024 by Muennighoff
4
KeyError: '### Model A: voyage-multilingual-2'
#24 opened Jul 30, 2024 by Muennighoff
Classification tab
#23 opened Jul 30, 2024 by Muennighoff
1
Make Clustering more intuitive
#22 opened Jul 30, 2024 by Muennighoff
Track time people take to vote
#20 opened Jul 30, 2024 by Muennighoff
3
Add leaderboard categories
#17 opened Jul 29, 2024 by Muennighoff
Corpus improvements
#16 opened Jul 27, 2024 by Muennighoff
Wikipedia corpus tables ignored
#15 opened Jul 27, 2024 by Muennighoff
1
Add personal leaderboard
#14 opened Jul 19, 2024 by Muennighoff
8
Add share button to Clustering arena
#4 opened Jun 24, 2024 by Muennighoff
Add share button to STS arena
#3 opened Jun 24, 2024 by Muennighoff
ProTip! Updated in the last three days: updated:>2024-10-01.