MINGYISU commited on
Commit
576cf58
·
verified ·
1 Parent(s): eb02c16

Update utils.py

Browse files
Files changed (1) hide show
  1. utils.py +8 -2
utils.py CHANGED
@@ -35,6 +35,12 @@ Building upon on **MMEB-V1**, **MMEB-V2** expands the evaluation scope to includ
35
  — Video Retrieval, Moment Retrieval, Video Classification, and Video Question Answering — and one task focused on visual documents, Visual Document Retrieval.
36
  This comprehensive suite enables robust evaluation of multimodal embedding models across static, temporal, and structured visual data settings.
37
 
 
 
 
 
 
 
38
  | [**📈Overview**](https://tiger-ai-lab.github.io/VLM2Vec/) | [**Github**](https://github.com/TIGER-AI-Lab/VLM2Vec)
39
  | [**📖MMEB-V2/VLM2Vec-V2 Paper**](https://arxiv.org/abs/2507.04590)
40
  | [**📖MMEB-V1/VLM2Vec-V1 Paper**](https://arxiv.org/abs/2410.05160)
@@ -42,8 +48,8 @@ This comprehensive suite enables robust evaluation of multimodal embedding model
42
  | [**Discord**](https://discord.gg/njyKubdtry) |
43
  """
44
 
45
- TABLE_INTRODUCTION = """***Important Notes: ***
46
- This is the MMEB-V1 leaderboard, which is now deprecated. MMEB-V1 is now the Image section of MMEB-V2, and the results on this leaderboard have been integrated into MMEB-V2 Image tab. For researchers relying on MMEB-V1, we recommend transitioning to MMEB-V2 for more comprehensive evaluation metrics and support. Thank you for your collaborations and understanding! \n"""
47
 
48
  LEADERBOARD_INFO = f"""
49
  ## Dataset Overview
 
35
  — Video Retrieval, Moment Retrieval, Video Classification, and Video Question Answering — and one task focused on visual documents, Visual Document Retrieval.
36
  This comprehensive suite enables robust evaluation of multimodal embedding models across static, temporal, and structured visual data settings.
37
 
38
+ **IMPORTANT NOTES Regarding the old MMEB-V1 leaderboard:**
39
+ MMEB-V1 is now part of the Image section of MMEB-V2, and the results on its leaderboard have been merged into the MMEB-V2 Image leaderboard.
40
+ We won't be accepting new submissions to the old V1 leaderboard anymore, and will remove it from this web page soon.
41
+ For researchers relying on MMEB-V1, we recommend transitioning to MMEB-V2 for more comprehensive evaluation metrics and support.
42
+ **On the V2 Image leaderboard, models from the old V1 leaderboard are missing detailed scores of each dataset. We hope the authors of the models on V1 leaderboard could rerun your models using our updated V2 pipeline, and provide us the scores sheet with the new format, so that we can make them consistent with the other models' formats.**
43
+
44
  | [**📈Overview**](https://tiger-ai-lab.github.io/VLM2Vec/) | [**Github**](https://github.com/TIGER-AI-Lab/VLM2Vec)
45
  | [**📖MMEB-V2/VLM2Vec-V2 Paper**](https://arxiv.org/abs/2507.04590)
46
  | [**📖MMEB-V1/VLM2Vec-V1 Paper**](https://arxiv.org/abs/2410.05160)
 
48
  | [**Discord**](https://discord.gg/njyKubdtry) |
49
  """
50
 
51
+ TABLE_INTRODUCTION = """***Important Notes:***
52
+ This is the old MMEB-V1 leaderboard, which is now deprecated and going to be removed from this web page soon. MMEB-V1 is now the Image section of MMEB-V2, and the results on this leaderboard have been integrated into MMEB-V2 Image section. For researchers relying on MMEB-V1, we recommend transitioning to MMEB-V2 for more comprehensive evaluation metrics and support. Thank you for your collaborations and understanding! \n"""
53
 
54
  LEADERBOARD_INFO = f"""
55
  ## Dataset Overview