[Task] Add support for VisualPuzzles

[yueqis]

Before you open a pull-request, please check if a similar issue already exists or has been closed before.

When you open a pull-request, please be sure to include the following

• A descriptive title: [Task] Add support for VisualPuzzles
• A detailed description

If you meet the lint warnings, you can use following scripts to reformat code.

pip install pre-commit
pre-commit install
pre-commit run --all-files

Thank you for your contributions!

Description of PR: Add support for VisualPuzzles

VisualPuzzles is a benchmark that targets visual reasoning while deliberately minimizing reliance on specialized knowledge. VisualPuzzles consists of 1168 diverse questions spanning five categories: algorithmic, analogical, deductive, inductive, and spatial reasoning. Each puzzle is labeled as easy, medium, or hard. All puzzles are multiple-choice questions with 4 options.

Two evaluation prompts are included, one utilizing CoT lmms_eval/tasks/VisualPuzzles/VisualPuzzles_cot.yaml, one without CoT lmms_eval/tasks/VisualPuzzles/VisualPuzzles_direct.yaml.

Arxiv: https://arxiv.org/abs/2504.10342
Huggingface Dataset: https://huggingface.co/datasets/neulab/VisualPuzzles
Project Webpage: https://neulab.github.io/VisualPuzzles/
Github Code: https://github.com/neulab/VisualPuzzles/tree/main