Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dataset option issue #5

Open
expapa opened this issue May 23, 2024 · 1 comment
Open

dataset option issue #5

expapa opened this issue May 23, 2024 · 1 comment

Comments

@expapa
Copy link

expapa commented May 23, 2024

Thanks for your contribution, nice work.

{'index': 22147, 'question': 'Please write a poem based on the image.', 'answer': 'J', 'A': "Promise the joy that I've longed to hear", 'B': 'The essence of what you could be', 'C': 'Space you fill in my memory', 'D': 'Visions of our time apart', 'E': 'Promise a whisper that I need to hear', 'F': 'Profound impact, in my youth', 'G': "Promise the stars that I've aimed to hold", 'H': 'Illusions of the paths we walk', 'I': 'Thoughts of our future to come', 'category': 'writing_poetry_from_image', 'l2-category': 'visual_captioning', 'split': 'VAL'}

The entire dataset has no option of 'J', but the answer of the question in this sample is 'J', which is incorrect for researchers to use, any idea why?

@KainingYing
Copy link
Collaborator

Thanks for your attention on MMT-Bench.

Thank you for your careful reminder. This was a error in our dataset, which we have now corrected. Additionally, we have checked the other samples to ensure there are no further mistakes. The updated dataset will be released shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants