Scientific LLM Benchmarks
GitHub
← All benchmarks
Physics & Astro· research-level-physics

CritPt

Argonne National Laboratory / UIUC (50+ physicists, 30+ institutions) · 2025

71 composite research-project challenges and 190 modular checkpoints authored by 50+ active physicists, auto-graded via numerical, symbolic, and code evaluations.

Physics
GitHub stars
Task type
open-ended
Modality
text
Access
open
Size
71 items
License
Metrics
accuracy
problem_id
Challenge_1_main
problem_type
main
problem_description
# Problem setup: Consider a quantum field theory with holographic dual. Under a Weyl transformation, the boundary metric transforms as $\gamma_{\mu\nu}^{(0)}\to{\cal B}^{-2}(x)\gamma_{\mu\nu}^{(0)}$. The Weyl anomaly ${\cal A}_k$ of the theory in $2k$ dimensions appears in the transformation of the partition function: \begin{align} Z[\gamma_{\mu\nu}^{(0)}]\to e^{-{\cal A}_k}Z[{\cal B}(x)^{-2}\gamma^{(0)}]. \end{align} This can be computed by evaluating the on-shell action of the bulk gravitational theory. The holographic Weyl anomaly in $d\leqslant8$ can be express using the following quant …
code_template
def answer(): r""" Return coefficients of the terms. Input ---------- None Output ---------- coeffs: list[float], the coefficients of terms in $X^{(4)}$, in the order given in the problem """ # ------------------ FILL IN YOUR RESULTS BELOW ------------------ coeffs = ... # --------------------------------------------------------------- return coeffs
answer_code
def answer(): r""" Return coefficients of the terms. Input ---------- None Output ---------- coeffs: list[float], the coefficients of terms in $X^{(4)}$, in the order given in the problem """ # ------------------ FILL IN YOUR RESULTS BELOW ------------------ coeffs = ... # --------------------------------------------------------------- return coeffs
metadata_notebook_path
data/public_test_challenges/Challenge_1.ipynb
metadata_problem_setup
# Problem setup: Consider a quantum field theory with holographic dual. Under a Weyl transformation, the boundary metric transforms as $\gamma_{\mu\nu}^{(0)}\to{\cal B}^{-2}(x)\gamma_{\mu\nu}^{(0)}$. The Weyl anomaly ${\cal A}_k$ of the theory in $2k$ dimensions appears in the transformation of the partition function: \begin{align} Z[\gamma_{\mu\nu}^{(0)}]\to e^{-{\cal A}_k}Z[{\cal B}(x)^{-2}\gamma^{(0)}]. \end{align} This can be computed by evaluating the on-shell action of the bulk gravitational theory. The holographic Weyl anomaly in $d\leqslant8$ can be express using the following quant …
problem_id
Challenge_10_main
problem_type
main
problem_description
# Problem setup: In order to introduce torsion to the system, one can use the first-order formulation of general relativity. We define a local reference frame at each point of the $(3+1)$-dimensional manifold $\mathcal{M}$, the tetrad $e^A_\mu$, such that the metric can be written as $g_{\mu\nu}=e^A_\mu e^B_\nu \eta_{AB}$, where $\eta_{AB}$ is the flat Minkowski metric on the internal space of coordinates. The internal indices, denoted by the Latin alphabet, also run from $0$ to $3$ just like the spacetime ones. The metrics $g_{\mu\nu}$ and $\eta_{AB}$ can raise or lower the spacetime and inte …
code_template
def answer(): r""" Return the number of e-folds achieved at $t = 25000$. Inputs ---------- None Outputs ---------- efolds: float, number of e-folds at $t = 25000$ """ # ------------------ FILL IN YOUR RESULTS BELOW ------------------ efolds = ... # --------------------------------------------------------------- return efolds
answer_code
def answer(): r""" Return the number of e-folds achieved at $t = 25000$. Inputs ---------- None Outputs ---------- efolds: float, number of e-folds at $t = 25000$ """ # ------------------ FILL IN YOUR RESULTS BELOW ------------------ efolds = ... # --------------------------------------------------------------- return efolds
metadata_notebook_path
data/public_test_challenges/Challenge_10.ipynb
metadata_problem_setup
# Problem setup: In order to introduce torsion to the system, one can use the first-order formulation of general relativity. We define a local reference frame at each point of the $(3+1)$-dimensional manifold $\mathcal{M}$, the tetrad $e^A_\mu$, such that the metric can be written as $g_{\mu\nu}=e^A_\mu e^B_\nu \eta_{AB}$, where $\eta_{AB}$ is the flat Minkowski metric on the internal space of coordinates. The internal indices, denoted by the Latin alphabet, also run from $0$ to $3$ just like the spacetime ones. The metrics $g_{\mu\nu}$ and $\eta_{AB}$ can raise or lower the spacetime and inte …

Real rows from the Hugging Face datasets server · long values truncated