Scientific LLM Benchmarks
GitHub
← All benchmarks
General· multimodal-STEM

EMMA

CUHK / Microsoft · 2025

2,788 problems demanding cross-modal reasoning across mathematics, physics, chemistry, and coding for multimodal models.

ChemistryPhysicsMathematics
GitHub stars
Task type
VQA
Modality
multimodal
Access
open
Size
2,788 items
License
Metrics
accuracy
pid
chem_1
question
<image_1> An 'arrow-pushing' diagram is a common type of chemical image used to illustrate electron flow in mechanistic steps. The transition-state structure undergoes changes after the electron has been relocated or reacted. Which of the following options shows the structure after the change? Note: Disregard the arrows in the options. The selected option should represent the immediate product of the next step, not the final product of the reaction.
options
<image_2> <image_3> <image_4> <image_5>
answer
B
image_1
<image/binary>
image_2
<image/binary>
image_3
<image/binary>
image_4
<image/binary>
image_5
<image/binary>
subject
Chemistry
category
Reaction Simulation Pro
source
new_annotated
type
Multiple choice
pid
chem_2
question
<image_1> An 'arrow-pushing' diagram is a common type of chemical image used to illustrate electron flow in mechanistic steps. The transition-state structure undergoes changes after the electron has been relocated or reacted. Which of the following options shows the structure after the change? Note: Disregard the arrows in the options. The selected option should represent the immediate product of the next step, not the final product of the reaction.
options
<image_2> <image_3> <image_4> <image_5>
answer
A
image_1
<image/binary>
image_2
<image/binary>
image_3
<image/binary>
image_4
<image/binary>
image_5
<image/binary>
subject
Chemistry
category
Reaction Simulation Pro
source
new_annotated
type
Multiple choice

Real rows from the Hugging Face datasets server · long values truncated