Scientific LLM Benchmarks
GitHub
← All benchmarks
Math· multimodal

MathVerse

CUHK MMLab / Shanghai AI Lab · 2024

2,612 visual math problems in six diagram/text variants probing whether MLLMs truly interpret figures.

Mathematics
GitHub stars
Task type
VQA
Modality
multimodal
Access
open
Size
3,940 items
License
MIT
Metrics
accuracy
sample_index
1
problem_index
1
problem_version
Text Dominant
question
As shown in the figure, in triangle ABC, it is known that angle A = 80.0, angle B = 60.0, point D is on AB and point E is on AC, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°
image
<image/binary>
answer
D
question_type
multi-choice
metadata
{"split":"testmini","source":"GeoQA","subject":"Plane Geometry","subfield":"Angle"}
query_wo
Please directly answer the question and provide the correct option letter, e.g., A, B, C, D. Question: As shown in the figure, in triangle ABC, it is known that angle A = 80.0, angle B = 60.0, point D is on AB and point E is on AC, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°
query_cot
Please first conduct reasoning, and then answer the question and provide the correct option letter, e.g., A, B, C, D, at the end. Question: As shown in the figure, in triangle ABC, it is known that angle A = 80.0, angle B = 60.0, point D is on AB and point E is on AC, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°
question_for_eval
As shown in the figure, in triangle ABC, it is known that angle A = 80.0, angle B = 60.0, point D is on AB and point E is on AC, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°
sample_index
2
problem_index
1
problem_version
Text Lite
question
As shown in the figure, it is known that angle A = 80.0, angle B = 60.0, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°
image
<image/binary>
answer
D
question_type
multi-choice
metadata
{"split":"testmini","source":"GeoQA","subject":"Plane Geometry","subfield":"Angle"}
query_wo
Please directly answer the question and provide the correct option letter, e.g., A, B, C, D. Question: As shown in the figure, it is known that angle A = 80.0, angle B = 60.0, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°
query_cot
Please first conduct reasoning, and then answer the question and provide the correct option letter, e.g., A, B, C, D, at the end. Question: As shown in the figure, it is known that angle A = 80.0, angle B = 60.0, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°
question_for_eval
As shown in the figure, it is known that angle A = 80.0, angle B = 60.0, DE parallel BC, then the size of angle CED is () Choices: A:40° B:60° C:120° D:140°

Real rows from the Hugging Face datasets server · long values truncated