Scientific LLM Benchmarks
GitHub
← All benchmarks
General· multimodal

MMMU

IN.AI / University of Waterloo / OSU · 2023

11.5K college-level multimodal questions across six disciplines and 30 subjects.

GitHub stars
Task type
VQA
Modality
multimodal
Access
open
Size
11,550 items
License
Apache-2.0
Metrics
accuracy
id
test_Accounting_1
question
Brahma Industries sells vinyl replacement windows to home improvement retailers nationwide. The national sales manager believes that if they invest an additional $25,000 in advertising, they would increase sales volume by 10,000 units. <image 1> What is the total contribution margin?
options
['$759,000', '$1,138,500', '$714,500', '$44,500']
image_1
<image/binary>
img_type
['Tables']
answer
A
topic_difficulty
Medium
question_type
multiple-choice
subfield
Managerial Accounting
id
test_Accounting_2
question
Let us compute for the missing amounts under work in process inventory, what is the cost of goods manufactured? <image 1>
options
['$20,506', '$21,506', '$22,506', '$23,506']
image_1
<image/binary>
img_type
['Tables']
answer
B
topic_difficulty
Medium
question_type
multiple-choice
subfield
Managerial Accounting

Real rows from the Hugging Face datasets server · long values truncated