Skip to content

Commit

Permalink
Merge pull request #169 from EvolvingLMMs-Lab/pufanyi/fix_small_typo
Browse files Browse the repository at this point in the history
Fix a small typo for live_bench
  • Loading branch information
Luodian authored Jul 27, 2024
2 parents 5411925 + d0a42ae commit 04009b2
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lmms_eval/evaluator.py
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,7 @@ def evaluate(
# hack: remove image columns to speed avoid loading images and speed up postprocessing
# reason: doc_iterator will actually load image if it's in the doc.
docs = task.test_docs() if task.has_test_docs() else task.validation_docs()
if "d170" not in task_name and "dc100" not in task_name and "dc200" not in task_name and "llava_wilder" not in task_name and "livebench" not in task_name and "wildvision" not in task_name:
if "d170" not in task_name and "dc100" not in task_name and "dc200" not in task_name and "llava_wilder" not in task_name and "live_bench" not in task_name and "wildvision" not in task_name:
remove_cols = []
features = docs.features
# If it is an Image instance or a Sequence of Image instance. Remove it
Expand Down

0 comments on commit 04009b2

Please sign in to comment.