Scientific LLM Benchmarks
GitHub
← All benchmarks
Math· formal-proof

CombiBench

Moonshot AI / Numina · 2025

100 competition combinatorics problems formalized in Lean4, a domain underrepresented by existing formal benchmarks.

Mathematics
GitHub stars
Task type
proof
Modality
code
Access
open
Size
100 items
License
MIT
Metrics
pass@16
theorem_name
hackmath_1
natural_language
How many ways can a teacher select a group of 6 students to sit in the front row if the class has 13 students?
answer
1716
source
https://www.hackmath.net/en/word-math-problems/combinatorics
tag
hackmath
formal_statement
import Mathlib abbrev hackmath_1_solution : ℕ := sorry theorem hackmath_1 (sols : Finset (Fin 13 → Fin 2)) (h_sols : ∀ f, f ∈ sols ↔ ((List.ofFn f).count 0 = 6)) : sols.card = hackmath_1_solution := by sorry
theorem_name
hackmath_2
natural_language
There are 8 athletes participating in a sprint competition. The referee needs to select 3 athletes and assign them specific rankings (first place, second place, and third place). How many different arrangements are possible?
answer
336
source
https://www.hackmath.net/en/word-math-problems/combinatorics
tag
hackmath
formal_statement
import Mathlib abbrev hackmath_2_solution : ℕ := sorry theorem hackmath_2 (sols : Finset (Fin 8 → Fin 4)) (h_sols : ∀ f, f ∈ sols ↔ ((List.ofFn f).count 0 = 1) ∧ ((List.ofFn f).count 1 = 1) ∧ ((List.ofFn f).count 2 = 1)) : sols.card = hackmath_2_solution := by sorry

Real rows from the Hugging Face datasets server · long values truncated