Skip to content

Conversation

@ashun989
Copy link
Contributor

@ashun989 ashun989 commented Apr 22, 2025

Fix the bug: The images blending within the same batch leads to the performance degradation of Qwen2.5_VL when the batch size increases

After fixing:

  • model: Qwen/Qwen2.5-VL-3B-Instruct
  • task: vqav2_val_lite
  • results:
Batch size Performance Total Time(s)
1 0.7478 120.2
2 0.7490 109.9
4 0.7466 98.9

Copy link
Contributor

@Luodian Luodian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quite nice, thank you!

@Luodian Luodian merged commit 93e1ac9 into EvolvingLMMs-Lab:main Apr 23, 2025
1 check passed
dadwadw233 pushed a commit to dadwadw233/lmms-eval that referenced this pull request Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants