add task MMVet-v2

[lingfengren]

MM-Vet v2 is a extension of MMVet, which includes a new VL capability called "image-text sequence understanding", evaluating models' ability to process VL sequences.
paper: https://arxiv.org/abs/2408.00765
github: https://github.com/yuweihao/MM-Vet/tree/main/v2

Since the sample may have multiple images, it is provided mmvetv2_group_img.yaml for those models that cannot handle multiple image inputs.

[pufanyi]

Hi! Thank you so much for your contribution!!!!

It looks good to me! However, is it possible to add a simple comment before mmvetv2_group_img.yaml, something like (if I understood correctly):

# This task differs from mmvet2 in that it combines multiple images into one, to accommodate models that cannot accept multiple images as input.

Thanks!!!

[lingfengren]

Hi! Thank you so much for your contribution!!!!

It looks good to me! However, is it possible to add a simple comment before mmvetv2_group_img.yaml, something like (if I understood correctly):

# This task differs from mmvet2 in that it combines multiple images into one, to accommodate models that cannot accept multiple images as input.

Thanks!!!

Ok, I'll add this comment later

[pufanyi]

Thanks!!!

[Luodian]

Hi I think the evaluation still has some errors?

[lingfengren]

Hi I think the evaluation still has some errors?

It seems that len of image_list is zero, the images should be stored in doc follow with image name as key.
This works fine for me.

if __name__ == "__main__":
    doc: dict[str, str] = {
        "question": "Describe the two pointed objects in the image.<IMG>v2_355_0.png<IMG>image<IMG>v2_9_0.png",
    }
    img_names = get_images_tokens(doc["question"])
    for img_name in img_names:
        doc[img_name] = Image.open(
            "path2mmvetv2/mm-vet-v2/images/" + img_name
        ).convert("RGB")
    print(mmvet_doc_to_visual(doc))
    print(mmvet_group_img_doc_to_visual(doc))

[pufanyi]

It seems that it is because there are 2 doc_to_visual functions in utils.py:

lmms-eval/lmms_eval/tasks/mmvetv2/utils.py

Line 165 in d900496

def mmvet_doc_to_visual(doc):

lmms-eval/lmms_eval/tasks/mmvetv2/utils.py

Line 297 in d900496

def mmvet_doc_to_visual(doc):

I have fixed it with some other errors in #457