Skip to content

[Fix] LiveBench 2409 #308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kcz358 merged 10 commits into from
Oct 9, 2024
Merged

[Fix] LiveBench 2409 #308

kcz358 merged 10 commits into from
Oct 9, 2024

Conversation

@pufanyi
Copy link
Collaborator

@pufanyi pufanyi commented Oct 9, 2024

Before you open a pull-request, please check if a similar issue already exists or has been closed before.

When you open a pull-request, please be sure to include the following

  • A descriptive title: [xxx] XXXX
  • A detailed description

Thank you for your contributions!

pufanyi added 8 commits October 5, 2024 20:07
@pufanyi
Refactor code formatting in setup.py, .gitignore, __init__.py, and da…
043fa69 
…ta_summary.ipynb
@pufanyi
Refactor DefaultWebsite class in website.py
eccc76c
@pufanyi
Refactor dataset name and limit processed images to 2
2cb32eb
@pufanyi
Refactor extract_infomation.py and update prompts
2930792
@pufanyi
update
9a20b96
@pufanyi
Refactor live_bench_2409.yaml and live_bench.yaml
052517c
@pufanyi
Refactor live_bench/utils_v2.py and question_finalizer.py
979202b
@pufanyi
Squashed commit of the following:
1dd348a 
commit 581a672
Author: Yinan He <[email protected]>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <[email protected]>

commit 99fcd39
Author: GIO PAIK <[email protected]>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (#305)

commit 6e1d747
Author: Li Bo <[email protected]>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303)

commit b838ff6
Author: Pu Fanyi <[email protected]>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <[email protected]>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (#296)
@pufanyi pufanyi requested a review from kcz358 October 9, 2024 07:02
@kcz358 kcz358 merged commit 39e1586 into main Oct 9, 2024
KairuiHu pushed a commit that referenced this pull request Oct 24, 2024
@pufanyi @KairuiHu
[Fix] LiveBench 2409 (#308)
3eefe54 
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

* Refactor DefaultWebsite class in website.py

* Refactor dataset name and limit processed images to 2

* Refactor extract_infomation.py and update prompts

* update

* Refactor live_bench_2409.yaml and live_bench.yaml

* Refactor live_bench/utils_v2.py and question_finalizer.py

* Squashed commit of the following:

commit 581a672
Author: Yinan He <[email protected]>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <[email protected]>

commit 99fcd39
Author: GIO PAIK <[email protected]>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (#305)

commit 6e1d747
Author: Li Bo <[email protected]>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303)

commit b838ff6
Author: Pu Fanyi <[email protected]>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <[email protected]>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (#296)

* fix
@pufanyi pufanyi deleted the pufanyi/live_bench_2409 branch December 28, 2024 07:15
MichalCiesiolka pushed a commit to MichalCiesiolka/lmms-eval-llmzszl that referenced this pull request Apr 3, 2025
@pufanyi
[Fix] LiveBench 2409 (EvolvingLMMs-Lab#308)
dfd97c2 
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

* Refactor DefaultWebsite class in website.py

* Refactor dataset name and limit processed images to 2

* Refactor extract_infomation.py and update prompts

* update

* Refactor live_bench_2409.yaml and live_bench.yaml

* Refactor live_bench/utils_v2.py and question_finalizer.py

* Squashed commit of the following:

commit 581a672
Author: Yinan He <[email protected]>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (EvolvingLMMs-Lab#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <[email protected]>

commit 99fcd39
Author: GIO PAIK <[email protected]>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (EvolvingLMMs-Lab#305)

commit 6e1d747
Author: Li Bo <[email protected]>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (EvolvingLMMs-Lab#303)

commit b838ff6
Author: Pu Fanyi <[email protected]>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (EvolvingLMMs-Lab#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <[email protected]>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (EvolvingLMMs-Lab#296)

* fix
dadwadw233 pushed a commit to dadwadw233/lmms-eval that referenced this pull request Apr 28, 2025
@pufanyi
[Fix] LiveBench 2409 (EvolvingLMMs-Lab#308)
7a56de8 
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

* Refactor DefaultWebsite class in website.py

* Refactor dataset name and limit processed images to 2

* Refactor extract_infomation.py and update prompts

* update

* Refactor live_bench_2409.yaml and live_bench.yaml

* Refactor live_bench/utils_v2.py and question_finalizer.py

* Squashed commit of the following:

commit e84f343
Author: Yinan He <[email protected]>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (EvolvingLMMs-Lab#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <[email protected]>

commit 87641e3
Author: GIO PAIK <[email protected]>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (EvolvingLMMs-Lab#305)

commit af87cc5
Author: Li Bo <[email protected]>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (EvolvingLMMs-Lab#303)

commit 3680ae4
Author: Pu Fanyi <[email protected]>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (EvolvingLMMs-Lab#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit ded426a
Author: Zhijian Liu <[email protected]>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (EvolvingLMMs-Lab#296)

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kcz358 kcz358 kcz358 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pufanyi @kcz358