[Feat] Add support for evaluation of Oryx models #276
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Hi thanks for the contribution! But wondering if it can be merged into one file for both image & video operations, since there are plenty of duplicated code. |
Contributor
Author
|
Working on it! |
Contributor
Author
|
Add evaluation results using lmms-eval |
Collaborator
|
LGTM after merging into the same model. May I ask do you have a screenshot of the evaluation result using current code and aligns with the paper? |
Contributor
|
@dongyh20 Hi Yuhao, can you share one screenshot to show that the terminal results of oryx model matched in paper numbers? |
Contributor
Author
|
working on it today |
Contributor
Author
|
Tested with Oryx-7B and Oryx-34B models |
KairuiHu
pushed a commit
that referenced
this pull request
Oct 24, 2024
* support Oryx models * update oryx model * update oryx * update oryx --------- Co-authored-by: dongyh20 <[email protected]>
MichalCiesiolka
pushed a commit
to MichalCiesiolka/lmms-eval-llmzszl
that referenced
this pull request
Apr 3, 2025
* support Oryx models * update oryx model * update oryx * update oryx --------- Co-authored-by: dongyh20 <[email protected]>
dadwadw233
pushed a commit
to dadwadw233/lmms-eval
that referenced
this pull request
Apr 28, 2025
* support Oryx models * update oryx model * update oryx * update oryx --------- Co-authored-by: dongyh20 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add Orxy models into lmms-eval/models to support image and video evaluations for Oryx