Scientific LLM Benchmarks
GitHub
← All benchmarks
General· knowledge

SciEval

Fudan University (OpenDFM) · 2023

~18K multi-level questions testing scientific knowledge across chemistry, physics, and biology.

BiologyChemistryPhysics
GitHub stars
Task type
MCQ
Modality
text
Access
open
Size
18,000 items
License
Metrics
accuracy
category
biology
cot_prompt
Given a question and four options, please select the right answer. Your answer should be "A", "B", "C" or "D".
prompt
Given a question and four options, please select the right answer. Your answer should be "A", "B", "C" or "D". Please directly give the answer without any explanation.
task_name
MedQA
answer
D
type
multiple-choice
question
A 62-year-old woman comes to the physician because of a 6-month history of progressive pain and stiffness of the fingers of her right hand. The stiffness is worse at the end of the day. She recently retired after working for 28 years as a typist at a data entry firm. Physical examination shows swelling, joint-line tenderness, and decreased range of motion of the first metacarpophalangeal joint and the distal interphalangeal joints of the right hand. Discrete, hard, mildly tender nodules are palpated over the 2nd and 4th distal interphalangeal joints of the right hand. An x-ray of her right han …
ability
Knowledge Application
category
biology
cot_prompt
Given a question and four options, please select the right answer. Your answer should be "A", "B", "C" or "D".
prompt
Given a question and four options, please select the right answer. Your answer should be "A", "B", "C" or "D". Please directly give the answer without any explanation.
task_name
MedQA
answer
C
type
multiple-choice
question
A 14-year-old girl is brought to the physician by her father because of fever, chills, abdominal pain, and profuse non-bloody diarrhea. Her symptoms began one week ago, when she had several days of low-grade fever and constipation. She returned from Indonesia 2 weeks ago, where she spent the summer with her grandparents. Her temperature is 39.3°C (102.8°F). Examination shows diffuse abdominal tenderness and mild hepatosplenomegaly. There is a faint salmon-colored maculopapular rash on her trunk and abdomen. Which of the following is the most likely causal organism? A. Giardia lamblia B. Schis …
ability
Knowledge Application

Real rows from the Hugging Face datasets server · long values truncated