Scientific LLM Benchmarks
GitHub
← All benchmarks
General· knowledge

SciKnowEval

Zhejiang University (HICAI) · 2024

Tens of thousands of problems across five cognitive levels in biology, chemistry, physics, and materials.

BiologyChemistryPhysicsMaterials
GitHub stars
Task type
QA
Modality
text
Access
open
Size
28,392 items
License
MIT
Metrics
accuracy, word2vec similarity, GPT-4o score
prompt
{"default":"Given a question and four options, please select the right answer. Your answer should be \"A\", \"B\", \"C\" or \"D\". Please directly give the answer without any explanation."}
question
In the context of anaerobic metabolism, which of the following correctly describes the function of S- or Se-methyltransferases in relation to thiol and selenol metabolites?
type
mcq-4-choices
domain
Biology
details
{"level":"L1","task":"literature_multi_choice_question","subtask":"BioRxiv_QA","source":"BioRxiv"}
answerKey
B
choices
{"text":["They regulate the transport of thiol and selenol metabolites","They (de)methylate thiol and selenol metabolites","They break down thiol and selenol metabolites","They catalyze the formation of thiol and selenol metabolites"],"label":["A","B","C","D"]}
prompt
{"default":"Given a question and four options, please select the right answer. Your answer should be \"A\", \"B\", \"C\" or \"D\". Please directly give the answer without any explanation."}
question
Which of the following mechanisms is NOT associated with the modulatory effects of farnesol on C. auris?
type
mcq-4-choices
domain
Biology
details
{"level":"L1","task":"literature_multi_choice_question","subtask":"BioRxiv_QA","source":"BioRxiv"}
answerKey
B
choices
{"text":["Inhibition of growth of multidrug resistant C. auris isolates","Upregulation of biofilm-associated genes","Downregulation of efflux pump-associated genes","Modulation of ABC efflux transporters of C. auris"],"label":["A","B","C","D"]}

Real rows from the Hugging Face datasets server · long values truncated