Scientific LLM Benchmarks
GitHub
← All benchmarks
General· knowledge

MMLU-Pro

University of Waterloo (TIGER-Lab) · 2024

12K reasoning-focused ten-option questions across 14 academic and STEM domains.

GitHub stars
Task type
MCQ
Modality
text
Access
open
Size
12,102 items
License
MIT
Metrics
accuracy
question_id
70
question
Typical advertising regulatory bodies suggest, for example that adverts must not: encourage _________, cause unnecessary ________ or _____, and must not cause _______ offence.
options
Safe practices, Fear, Jealousy, Trivial Unsafe practices, Distress, Joy, Trivial Safe practices, Wants, Jealousy, Trivial Safe practices, Distress, Fear, Trivial Unsafe practices, Wants, Jealousy, Serious Safe practices, Distress, Jealousy, Serious Safe practices, Wants, Fear, Serious Unsafe practices, Wants, Fear, Trivial
answer
I
answer_index
8
category
business
src
ori_mmlu-business_ethics
question_id
71
question
Managers are entrusted to run the company in the best interest of ________. Specifically, they have a duty to act for the benefit of the company, as well as a duty of ________ and of _______.
options
Shareholders, Diligence, Self-interest Shareholders, Self-interest, Care and Skill Stakeholders, Care and skill, Self-interest Stakeholders, Diligence, Care and Skill Customers, Care and Skill, Diligence Shareholders, Care and Skill, Diligence Shareholders, Self-interest, Diligence Employees, Care and Skill, Diligence
answer
F
answer_index
5
category
business
src
ori_mmlu-business_ethics

Real rows from the Hugging Face datasets server · long values truncated